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Abstract 

This  paper  explores  a  topological  perspective  of  planning  in  the  presence  of  uncertainty, 
focusing  on  tasks  specified  by  goal  states  in  discrete  spaces.  The  paper  introduces  strategy 
complexes.  A  strategy  complex  is  the  collection  of  all  plans  for  attaining  all  goals  in  a 
given  space.  Plans  are  like  jigsaw  pieces.  Understanding  how  the  pieces  fit  together  in  a 
strategy  complex  reveals  structure.  That  structure  characterizes  the  inherent  capabilities 
of  an  uncertain  system.  By  adjusting  the  jigsaw  pieces  in  a  design  loop,  one  can  build 
systems  with  desired  competencies. 

The  paper  draws  on  representations  from  combinatorial  topology,  Markov  chains,  and 
polyhedral  cones.  Triangulating  between  these  three  perspectives  produces  a  topological 
language  for  describing  concisely  the  capabilities  of  uncertain  systems,  analogous  to 
concepts  of  reachability  and  controllability  in  other  disciplines.  The  major  nouns  in  this 
language  are  topological  spaces. 

Three  key  theorems  (numbered  1,  11,  20  in  the  paper)  illustrate  the  sentences  in  this 
language:  (a)  Goal  Attainability:  There  exists  a  strategy  for  attaining  a  particular  goal 
from  anywhere  in  a  system  if  and  only  if  the  strategy  complex  of  a  slightly  modified  system 
is  homotopic  to  a  sphere,  (b)  Full  Controllability:  A  system  can  move  between  any  two 
states  despite  control  uncertainty  precisely  when  its  strategy  complex  is  homotopic  to  a 
sphere  of  dimension  two  less  than  the  number  of  states,  (c)  General  Structure:  Any 
system’s  strategy  complex  is  homotopic  to  the  product  of  a  spherical  part,  modeling  full 
controllability  on  subspaces,  and  a  general  part,  modeling  adversarial  capabilities. 

The  paper  contains  a  number  of  additional  results  required  as  stepping  stones,  along 
with  many  examples.  The  paper  provides  algorithms  for  computing  the  key  structures 
described.  Finally,  the  paper  shows  that  some  interesting  questions  are  hard.  For  instance, 
it  is  AP-complete  to  determine  the  most  precisely  attainable  goal  of  a  system  with  perfect 
sensing  but  uncertain  control. 
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reflect  the  position  or  the  policy  of  the  U.S.  Government.  No  official  endorsement  should  be  inferred. 
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1  Introduction 


1.1  Planning  Manipulation  Strategies  with  Uncertainty 

Two  themes  pervade  the  last  four  decades  of  research  in  robotic  manipulation.  The  first  is  the 
difficulty  of  uncertainty,  the  second  is  a  solution  by  discrete  modeling.  The  jamming  diagram 
appearing  in  Whitney’s  classic  paper  on  peg-in-hole  assembly  [77] ,  grounded  in  extensive  work 
at  the  C.S.  Draper  Laboratory  [62]  in  the  1970s,  illustrates  both  themes:  (i)  The  precise 
location  and  motion  of  the  peg  are  uncertain,  (ii)  The  contact  state  of  the  peg  naturally 
discretizes  the  assembly  space.  By  analyzing  this  discretized  space,  [77]  produces  an  automatic 
strategy  for  assembly  that  is  robust  in  the  presence  of  uncertainty. 

As  this  early  example  demonstrates,  discretization  appears  in  manipulation  research  to 
simplify,  but  not  artificially  so.  Instead,  the  mechanics  of  a  problem  often  generate  natural 
discrete  states  even  when  the  initial  rendering  of  the  problem  is  continuous.  Contact  modes, 
describing  which  features  of  multiple  objects  are  in  contact  and  how  are  they  sliding  relative 
to  each  other,  are  particularly  common  methods  of  discretization.  As  further  illustration,  the 
cooperative  manipulation  strategies  of  [27]  are  based  on  discrete  states  representing  regions  of 
configuration  space  over  which  the  frictional  contact  mechanics  of  the  robot  palms  and  the  part 
are  invariant.  Discrete  states  may  also  capture  higher-order  information,  perhaps  modeling 
sensing  uncertainty.  For  instance,  in  the  part  orienters  of  [29,  72,  37,  30],  the  discrete  states 
considered  by  the  motion  planners  were  sets  of  underlying  contact  states  of  the  parts  being 
oriented.  In  this  manner,  sensing  uncertainty  itself  contributes  to  the  definition  of  state  [5,  54]. 

Delineating  relevant  states  is  only  part  of  discrete  modeling.  One  must  also  describe 
transitions  between  states,  with  the  aim  of  synthesizing  robust  manipulation  strategies.  Anyone 
who  has  ever  programmed  a  robot  manipulator  quickly  learns  never  to  expect  a  particular 
grasping  operation  or  a  particular  assembly  trajectory  to  succeed.  Instead,  transitions  between 
states  are  generally  uncertain,  due  both  to  underlying  control  uncertainty  and,  in  the  discrete 
case,  to  approximation  effects.  As  a  result,  the  discrete  modeling  of  a  manipulation  problem 
does  not  define  a  standard  directed  graph,  but  rather  a  nondeterministic  or  stochastic  graph,  in 
which  commanded  motions  may  have  any  one  of  several  possible  outcomes.  Consequently,  one 
must  program  robots  by  thinking  in  terms  of  sets  of  possible  motions  not  individual  trajectories. 
The  preimage  methodology  of  [56]  emphasizes  this  point  and  shows  how  to  generate  robust 
strategies  in  a  fairly  general,  continuous,  setting.  In  the  discrete  setting,  that  methodology  is 
akin  to  Bellman-style  backchaining  [4].  Recent  manipulation  results  [42,  43]  demonstrate  the 
utility  of  these  ideas  in  stochastic  settings. 

1.2  Understanding  System  Capabilities 

The  description  of  planning  above  is  highly  operational.  One  can  almost  turn  a  crank:  Given 
a  manipulation  task,  create  a  discrete  representation  of  the  task  based  on  the  task  mechanics 
and  uncertainties,  then  backchain  from  the  task  goal.  Missing  is  an  understanding  of  the  full 
capabilities  of  such  a  discrete  representation.  Operationally,  one  can  report  whether  a  planner 
found  a  plan  to  perform  some  task,  but  one  does  not  understand  intrinsically  when  there  should 
be  a  plan  and  what  failure  to  find  a  plan  really  means.  It  would  be  good  to  understand  what 
the  remedy  for  a  failure  might  be,  for  instance,  more  precise  sensing,  more  precise  control,  a 
change  in  the  task  goal,  or  perhaps  even  a  remodeling  of  the  task  description  and  discretization. 
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Our  field  does  not  even  have  an  adequate  language  to  describe  the  issues.  In  control  theory, 
terms  such  as  “controllable”,  “unstable”,  and  “limit  cycle”  are  common;  in  graph  theory, 
“strongly  connected”,  “source”  and  “sink”;  in  Markov  chains,  “recurrent  class”,  “transient” 
and  “absorbing”.  In  different  languages  and  in  different  settings,  these  terms  describe  similar 
concepts,  significant  for  characterizing  system  capabilities. 

Such  terms  also  offer  standards  by  which  to  design  systems.  One  learns  how  to  create 
or  destroy  limit  cycles  in  control  theory,  for  example.  One  understands  that  a  finite  Markov 
chain  cannot  consist  exclusively  of  transient  states,  whereas  an  infinite  one  can.  All  effective 
engineering  fields  have  such  concise  descriptions  of  capabilities.  For  instance,  to  pick  an 
example  very  different  from  the  previous  ones,  in  computer  language  theory,  one  learns  that 
certain  context-free  grammars  are  easily  compilable  while  others  are  not. 

Planning  in  the  presence  of  uncertainty  has  no  such  concise  descriptions  of  capabilities  and 
thus  no  concise  design  standards. 

1.3  Topology 

A  thesis  of  this  paper  is  that  topology  provides  a  language  for  concise  descriptions  of  system 
capabilities.  In  order  to  develop  that  language,  we  follow  a  tack  common  in  engineering 
disciplines:  Given  a  method  for  solving  any  particular  problem  operationally,  one  should 
attempt  to  classify  problems  and  solutions  generally,  for  instance,  by  observing  similarities 
in  problems  and  exploiting  commonalities  of  solution.  In  the  case  of  planning  for  uncertain 
systems,  a  plan  seldom  exists  in  isolation;  instead  it  is  part  of  an  ensemble  of  interconnected 
strategies  for  accomplishing  an  ensemble  of  interconnected  goals  in  some  system. 

Topology  is  good  at  extracting  structure  from  such  an  interconnected  patchwork.  That 
structure  constitutes  our  language.  Moreover,  this  language,  as  we  will  see,  has  two  desirable 
traits: 

1.  The  topological  descriptors  abstract  away  details  of  particular  trajectories,  focusing 
instead  on  overall  system  capabilities. 

2.  The  topological  descriptors  are  consistent  with  existing  terminology  for  standard  directed 
graphs  and  Markov  chains,  while  generalizing  both. 

Our  search  specifically  for  a  topological  language  to  describe  system  capabilities  was 
motivated  by  Robert  Ghrist’s  technology  transfer  between  topology  and  robotics  [13,  36,  34,  35], 
by  Steven  LaValle’s  work  on  information  spaces  [38,  65,  74,  73],  and  by  workshops  on  topology 
and  robotics  organized  by  Michael  Farber  at  ETH  Zurich  in  2003  and  2006  [57]. 

1.4  Result  Flavor 

An  analogy  for  understanding  the  relationship  of  topology  to  planning  might  be  the  relationship 
of  linear  algebra  to  vectors  written  with  specific  coordinates.  Linear  algebra  provides  abstract 
techniques  for  representing  and  manipulating  vectors,  independent  of  coordinates.  Anything 
one  can  do  with  matrix  and  vector  notation  one  can  do  as  well  by  writing  out  arrays  of 
coordinates  and  manipulating  the  coordinates  directly.  Indeed,  ultimately  computations  in  a 
computer  must  work  with  numbers  in  some  coordinate  system.  However,  the  numbers  are  like 
trees  that  obscure  the  forest.  By  thinking  instead  at  the  abstract  level  of  linear  subspaces, 
kernels,  eigenvectors,  and  so  forth,  one  can  recognize  fundamental  structure.  How  easy  it  is 
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to  say  that  a  high-dimensional  positive  definite  system  always  decomposes  into  orthogonal 
one-dimensional  systems;  how  cumbersome  it  would  be  to  convey  that  truth  by  numbers  or 
coordinates  alone. 

This  perspective  has  precedent  in  other  areas  of  computer  science,  such  as  the  pioneering 
work  of  Herlihy  on  asynchronous  computation.  For  instance,  [40]  shows  that  an  asynchronous 
decision  task  has  a  wait-free  protocol  if  and  only  if  there  exists  a  certain  color-preserving 
continuous  function  between  two  chromatic  simplicial  complexes  determined  by  the  task.  In 
other  words,  a  computational  problem  is  equivalent  to  a  topological  problem.  The  simplicial 
complexes  reflect  the  structure  of  the  input  and  output  spaces  of  the  task.  The  continuous 
function  is  a  topological  constraint  between  those  spaces  and  thus  a  constraint  on  solvability 
of  the  decision  task. 

In  topological  robotics,  Farber  initiated  a  line  of  work  to  describe  the  topological  complexity 
of  motion  planning  problems  on  a  space  in  terms  of  the  cohomology  of  that  space  [31].  This 
complexity  reflects  the  discontinuities  inherent  to  any  controller  that  maps  start  and  goal 
configurations  of  a  robot  to  trajectories  between  those  configurations. 

An  early  example  demonstrating  the  applicability  specifically  of  algebraic  topology  to 
robotics,  and  indeed  manipulation,  appears  in  the  work  of  Hopcroft  and  Wilfong  [41].  They 
use  the  Mayer-Vietoris  sequence  on  the  zeroth  and  first  homology  groups  to  study  contact 
connectivity.  A  key  result:  Under  certain  conditions  on  the  structure  of  configuration  space, 
if  there  is  a  motion  of  two  objects  starting  and  ending  with  the  objects  in  contact,  then  there 
exists  a  motion  throughout  which  the  objects  remain  in  contact. 

1.5  Spheres  as  Topological  Descriptors  of  Task  Solvability 

The  current  paper  focuses  on  tasks  that  may  be  specified  by  goal  states  in  some 
nondeterministic  or  stochastic  graph;  the  task  is  to  attain  some  goal  state  starting  from 
anywhere  within  the  graph.  A  key  result  (Theorem  1)  shows  that  such  a  task  has  a  guaranteed 
solution  if  and  only  if  a  certain  simplicial  complex  associated  with  the  task  is  homotopic  to 
a  sphere  of  a  certain  dimension.  This  special  result  leads  to  a  general  graph  controllability 
theorem  (Theorem  11)  that  characterizes  the  ability  of  a  system  to  achieve  any  goal  despite 
control  uncertainty,  again  in  terms  of  the  existence  of  a  certain  sphere.  Spheres  and  contractible 
spaces  are  much  like  the  linear  subspaces  and  trivial  kernels  in  our  earlier  analogy. 

These  results  are  motivated  by  similar  results  describing  the  structure  of  complete  directed 
graphs  [8]  and  strongly  connected  directed  graphs  [44] .  Indeed,  our  proof  techniques  build  on 
the  foundations  of  those  two  papers.  Of  additional  interest  is  the  extensive  analysis  in  [47]. 

It  is  worth  noting  that  the  domain  of  nondeterministic  graphs  is  much  richer  than  that  of 
directed  graphs.  It  turns  out  (Section  13)  that  every  finite  simplicial  complex  can  be  realized  via 
some  nondeterministic  graph.  This  correspondence  further  underscores  the  natural  connection 
of  topology  to  planning  in  the  presence  of  uncertainty. 

1.6  Contributions 

The  primary  contribution  of  this  paper  is  the  previously  unseen  structure  it  reveals  in  planning 
problems.  The  paper  shows  how  the  details  of  a  nondeterministic  or  stochastic  graph  may  be 
abstracted  away,  leaving  a  purely  topological  description  of  the  task:  One  may  reason  about 
task  solvability  by  thinking  in  terms  of  spheres  and  contractible  spaces. 
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A  second  contribution  is  the  introduction  of  particular  tools,  such  as  strategy  complexes.  A 
single  strategy  on  a  graph  is  an  eventually  convergent  nondeterministic  control  law  for  moving 
within  some  portion  of  the  graph.  A  strategy  complex  consists  of  all  possible  strategies  on  a 
graph.  Strategy  complexes  can  be  useful  for  reasoning  about  alternate  strategies  that  a  system 
might  need  if  a  selected  strategy  for  accomplishing  a  task  fails  unexpectedly. 

Third,  although  largely  beyond  the  scope  of  this  paper,  a  stream  of  ancillary  results 
flows  from  our  characterization  of  task  solvability,  among  them:  (i)  One  can  answer  purely 
topologically  the  question  whether  a  collection  of  actions  is  essential  to  solving  a  task,  (ii) 
One  can  recast  various  questions  about  tasks  into  measurements  by  the  Euler  characteristic, 
(iii)  One  discovers  that  all  strategies  for  accomplishing  a  goal  in  a  nondeterministic  graph 
must  overlap  a  particular  strategy  found  by  backchaining.  (iv)  One  observes  that  the  number 
of  strategies  for  attaining  a  single  goal  state  is  either  zero  or  odd  (this  is  the  topological 
equivalent  of  Yogi  Berra’s  direction:  “When  you  come  to  a  fork  in  the  road,  take  it.”). 

We  anticipate  yet  more  structure  will  be  discovered  in  planning  problems  via  the  lens  of 
topology.  Indeed,  every  key  topological  idea  we  have  looked  at  thus  far  has  had  some  significant 
meaning  when  recast  in  a  planning  context.  The  Combinatorial  Alexander  Dual  discussed  in 
Section  10  is  a  good  example. 

1.7  An  Underlying  Motivation 

A  longstanding  motivation  for  our  research  has  been  a  desire  to  understand  system  capabilities 
and  the  interplay  of  sensing  and  action.  That  motivation  is  rooted  in  a  long  line  of  work  on 
motion  planning  with  uncertainty  from  the  1980s  and  1990s  [9,  59,  60,  22,  10,  14,  12,  72,  15, 
16,  53,  33,  55,  11]  as  well  as  related  work  on  understanding  information  invariants,  such  as 
natural  tradeoffs  between  sensing  and  action  [70,  21,  18,  2,  19,  52,  26,  17,  45,  46].  The  tools  for 
characterizing  system  capabilities  discussed  in  Sections  9  and  10  provide  a  concise  topological 
language  for  making  such  comparisons. 

1.8  Broader  Context 

Applying  the  ideas  in  this  paper  to  robotics  problems  requires  front-end  work:  one  needs  to 
cast  a  robot  task  as  a  motion  planning  problem  within  some  graph  whose  transitions  are 
(potentially)  uncertain.  Much  of  robotics  research  over  the  past  three  decades  has  been 
devoted  precisely  to  such  reductions.  We  will  not  review  those  techniques  here,  but  point 
to  two  excellent  books:  [53]  and  [54].  For  instance,  in  manipulation,  one  generally  needs  to 
describe  the  geometry  and  mechanics  of  contact,  including  dealing  with  issues  such  as  friction, 
deformation,  and  so  forth.  In  many  instances,  these  analyses  are  fairly  local.  One  then  is  left 
with  the  task  of  combining  the  local  information  to  form  global  motion  planners.  The  current 
paper  presents  a  topological  perspective  of  that  second  phase. 

The  topological  perspective  likely  has  broader  applications  than  the  motivating  tasks  of 
robotic  manipulation.  Section  11.4  provides  a  small  example.  We  hope  that  the  techniques  in 
this  paper  will  be  useful  for  analyzing  planning  and  decision-making  problems  in  a  wide  variety 
of  domains,  namely  those  with  two  modular  phases:  a  front-end  discretization  phase,  followed 
by  a  planning  phase.  In  particular,  methods  such  as  those  discussed  in  Sections  9-15  should 
be  useful  not  only  for  planning  but  for  analyzing  and  fine-tuning  the  details  of  the  front-end 
phase.  Potentially  relevant  additional  domains  include  decisional  architecture  design,  temporal 
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reasoning,  and  hybrid  control  systems. 


1.9  Outline 

Our  basic  approach  is  to  replace  the  combinatorial  structure  of  a  discrete  uncertain  graph  with 
the  topology  of  a  collection  of  open  sets  in  Euclidean  space.  This  technique  is  well-established 
for  standard  directed  graphs  [8,  44].  We  show  how  to  generalize  the  approach  to  uncertain 
systems.  In  particular,  convergence  results  for  Markov  chains  appear  as  intersection  properties 
of  certain  affine  open  cones. 

The  resulting  topology  characterizes  system  capabilities.  We  indicate  how  to  use  such 
characterizations  for  designing  desired  systems,  for  analyzing  existing  systems,  and  for  general 
reasoning  about  motion  with  uncertainty. 

•  Section  1  discusses  core  approaches  to  planning  manipulation  strategies  and  how  these 
have  motivated  this  paper’s  topological  inquiry. 

•  Section  2  provides  intuition  for  understanding  nondeterministic  graphs  and  their  strategy 
complexes,  via  a  series  of  examples. 

•  Section  3  formally  introduces  nondeterministic  graphs  and  their  strategy  complexes. 

•  Section  4  introduces  loopback  graphs  and  complexes,  using  these  to  characterize  task 
solvability  by  the  existence  of  topological  spheres. 

•  Section  5  provides  background  on  the  topological  tools  appearing  in  this  paper. 

•  Section  6  adds  stochastic  actions  to  the  mix,  now  permitting  both  nondeterminism  and 
stochasticity  in  a  graph. 

•  Section  7  develops  a  connection  between  the  combinatorial  structure  of  plans  and  certain 
conical  open  sets  in  Euclidean  space,  useful  for  measuring  a  strategy’s  convergence  times. 

•  Section  8  characterizes  full  controllability  on  a  graph,  despite  control  uncertainty,  in 
terms  of  the  existence  of  a  sphere  of  dimension  two  less  than  the  number  of  states  in  the 
graph. 

•  Section  9  explores  methods  for  designing  systems  using  the  topological  tools  developed 
thus  far. 

•  Section  10  dualizes  the  earlier  spherical  characterizations,  leading  both  to  contractibility 
characterizations  and  two  new  simplicial  complexes:  the  source  complex  and  its  dual 
complex.  The  source  complex  consists  of  all  start  regions  of  convergent  plans,  the  dual 
complex  consists  of  all  potentially  unattainable  goals. 

•  Section  11  discusses  internal  structure:  how  to  combine  graphs,  how  to  simplify  graphs. 
An  implemented  example  illustrates  the  ideas. 

•  Section  12  contains  algorithms  to  implement  the  core  ideas  of  the  paper. 

•  Section  13  proves  two  realizability  theorems,  establishing  two  different  ways  in  which 
nondeterministic  graphs  are  just  like  finite  simplicial  complexes. 


•  Section  14  shows  that  ascertaining  an  uncertain  system’s  precision  is  JVP-complete.  The 
section  also  shows  that  two  natural  questions  one  might  ask  of  graphs  are  undecidable, 
based  on  the  undecidability  of  recognizing  contractibility. 

•  Section  15  discusses  several  different  ways  in  which  topological  thinking  illuminates 
planning  problems. 


2  An  Example:  Nondeterminism,  Cycles,  and  Strategies 


A 


Figure  1:  Direct  access  might  be  blocked. 


A 


B— »X 

C^X 


Figure  2:  Nondeterministic  graph  and  associated  strategy  complex  for  the  motions  of  Fig.  1. 

Legend:  Figures  in  this  paper  depict  nondeterministic  actions  in  two  ways:  In  a  graph,  as  directed 
edges  tied  together  by  a  circular  arc,  along  with  a  “dot”  inside  the  arc.  In  a  strategy  complex,  by 
multiple  outcomes  to  the  right  of  an  “arrow”  (— >). 

Imagine  an  ambulance  rescue  in  an  old  complicated  city,  perhaps  after  an  earthquake. 
There  are  many  opportunities  for  nondeterminism:  Entries  into  the  city  might  be  blocked, 
maps  might  be  wrong,  navigation  might  lead  to  circular  paths.  Let  us  focus  on  the  final  step 
in  which  the  ambulance  medics  must  pass  through  a  narrow  opening  to  reach  their  patient, 
as  in  Fig.  1.  Something  might  go  wrong,  a  collapse  of  some  sort,  forcing  the  rescuers  to  either 
side.  Let  us  suppose  the  rescuers  may  then  take  additional  steps  around  buildings  bounding 
the  original  narrow  opening  to  reach  their  patient. 

We  can  model  this  scenario  using  the  graph  in  the  left  panel  of  Fig.  2.  The  action  to  move 
from  A  (ambulance)  to  X  (patient)  might  non  deterministically  lead  to  X  but  perhaps  also  to  B 
or  C,  depending  on  whether  and  how  a  collapse  occurs  on  the  direct  path  from  A  to  X.  In  the 
example,  there  then  are  deterministic  actions  from  either  B  or  C  to  X.  Such  a  graph  is  essentially 
a  compressed  AND  / OR  graph  [3] . 
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We  can  now  represent  the  strategy  (or  control  law  or  plan)  just  described  as  a  solid  triangle 
(right  panel  of  Fig.  2).  The  vertices  of  the  triangle  are  the  individual  actions  to  be  executed  at 
any  particular  location  during  the  rescue  operation.  So,  for  instance,  the  strategy  says  “When 
at  location  B,  execute  the  action  B  — >  X.” 


Figure  3:  Actions  that  move  between  the  far  locations  B  and  C. 

Perhaps  it  is  also  possible  to  move  from  B  to  C,  as  in  Fig.  3,  and  vice-versa.  We  can  augment 
our  graph  to  include  these  actions  (left  panel  of  Fig.  4) 


Figure  4:  With  additional  (potentially  cycle-inducing)  actions,  the  strategy  complex  now 
contains  two  solid  tetrahedra. 

Once  we  have  the  additional  actions  B  — >  C  and  C  — >  B,  we  have  to  be  careful  not  to  include 
both  in  a  strategy.  Otherwise,  the  rescuers  (who  could  be  robots,  not  humans)  might  cycle 
forever  between  B  and  C,  never  reaching  their  patient  at  X.  Instead,  we  recognize  that  there 
now  exist  several  distinct  strategies  for  reaching  X .  These  can  be  represented  by  the  two  solid 
tetrahedra  in  the  right  panel  of  Fig.  4.  Observe  that  the  two  tetrahedra  intersect  in  a  triangle 
that  is  our  original  triangle  from  Fig.  2,  but  there  is  no  simplex  that  simultaneously  includes 
the  two  actions  B  — >  C  and  C  — >  B. 

Each  tetrahedron  represents  a  strategy  (or  control  law  or  plan)  consisting  of  actions  that 
may  be  executed  without  accidentally  creating  cycles  in  the  graph.  Each  triangle  or  edge 
or  vertex  of  one  of  these  tetrahedra,  formed  from  a  subset  of  the  actions  comprising  the 
tetrahedron,  also  represents  some  strategy  (perhaps  with  a  different  goal). 

The  semantics  of  the  top  tetrahedron  of  Fig.  4  are: 

-  When  at  A,  execute  the  action  A  — >  B,  X,  C. 

-  When  at  B,  execute  either  the  action  B  — >  X  or  the  action  B  — >  C. 

It  does  not  matter  which;  pick  one,  perhaps  nondeterministically. 

-  When  at  C,  execute  the  action  C  — >  X. 
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Nondeterminism  appears  in  both  the  outcomes  and  choices  of  the  strategy: 

1.  Nature  acts  as  an  adversary  during  execution  of  the  action  A  — ►  B,X,C,  making  the 
outcome  uncertain.  This  can  be  bad.  Fortunately,  in  the  example,  the  system  can 
compensate,  by  executing  additional  actions. 

2.  The  system  has  available  multiple  actions  at  location  B.  This  is  good;  it  provides 
redundancy,  alternate  paths.  The  system  can  either  leave  the  choice  of  which  action 
to  execute  at  state  B  open  or  it  can  select  a  particular  action.  The  tetrahedron  in  toto 
leaves  the  choice  open,  effectively  increasing  nondeterminism,  perhaps  handing  the  choice 
over  to  an  adversary.  Alternatively,  if  the  system  chooses  a  particular  action  at  state  B, 
it  is  effectively  picking  a  particular  triangular  face  of  the  tetrahedron  as  its  true  strategy. 


Terminology:  In  the  remainder  of  the  paper,  we  will  generally  speak  of  a  “strategy”  rather 
than  a  “plan”,  in  order  to  avoid  the  suggestion  that  strategies  are  created  or  discovered  by 
planners.  They  exist  independently  of  being  created  or  discovered. 

3  Nondeterministic  Graphs  and  Strategy  Complexes 

This  section  makes  precise  the  intuition  of  the  previous  section,  focusing  on  nondeterministic 
graphs;  Section  6  introduces  stochasticity. 

3.1  Basic  Definitions 
Nondeterministic  Graphs 

A  nondeterministic  graph  G  =  (V.  A)  is  a  set  of  states  V  and  a  collection  of  (nondeterministic) 
actions  A.  Each  A  e  A  is  a  nonempty  set  of  directed  edges  {(u,  it i ) ,  (v,  U2), . . .},  with  v  and  all 
Ui  in  V.  We  refer  to  v  as  A’s  source  and  to  each  m  as  a  (nondeterministic)  target  of  A.  If  A 
has  a  single  target,  A  is  also  said  to  be  deterministic. 

Action  A  may  be  executed  whenever  the  system  is  at  state  v.  When  action  A  is  executed, 
the  system  moves  from  state  v  to  one  of  the  targets  ut .  If  A  has  multiple  targets,  the  precise 
target  attained  is  not  known  ahead  of  time.  One  can  imagine  an  adversary  choosing  the  target. 

Distinct  actions  may  have  overlapping  or  identical  edge  sets.  (For  instance,  two  different 
global  motion  commands  might  have  the  same  effect  at  a  given  state.  We  could  introduce 
extra  notation  to  label  actions  with  names,  thereby  making  explicit  their  individual  identity, 
but  the  extra  notation  would  be  more  cumbersome  than  informative.) 

All  graphs,  sets  of  states,  actions,  and  collections  of  actions  in  this  paper  are  finite. 

Remark:  Nondeterministic  graphs  in  which  each  action  is  deterministic  and  no  two  actions 
have  the  same  edge  set  are  equivalent  to  standard  directed  graphs. 

Terminology:  We  speak  of  a  graph  “state”,  reserving  the  term  “vertex”  for  singleton 
simplices  in  simplicial  complexes. 
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Acyclic  Graphs  and  Subgraphs 

Suppose  G  =  {V,  A)  is  a  nondeterministic  graph.  A  possible  path  of  length  k  in  G  is  a 
sequence  of  states  vq,  v\,  . . . ,  in  V  such  that  (vi,Vi+ 1)  £  A%,  with  each  At  an  action  in 
A,  for  i  =  0, . . . ,  k  —  1.  G  is  acyclic  if  none  of  its  possible  paths  have  vq  =  Vk  with  k  >  1. 

A  nondeterministic  subgraph  H  =  (W.  B )  of  G  is  a  nondeterministic  graph  in  its  own  right 
such  that  W  C  V  and  BCd.  In  particular,  any  BAA  defines  a  nondeterministic  subgraph 
Hb  =  (V,  B)  of  G.  We  will  say  that  a  collection  of  actions  B  C  A  is  acyclic  if  its  induced 
subgraph  Hq  is  acyclic. 

Simplicial  Complexes 

An  (abstract)  simplicial  complex  S  is  a  collection  of  finite  sets,  such  that  if  a  is  in  S  then  so  is 
every  subset  of  a  [61].  The  elements  of  S  are  called  simplices ;  the  elements  of  a  simplex  and 
singleton  simplices  are  both  called  vertices.  Sometimes  one  refers  to  the  underlying  vertex  set 
of  a  simplicial  complex  S,  which  contains  all  the  vertices  of  S  and  may  contain  vertices  that 
could  be  present  in  S  even  if  they  happen  not  to  be.  The  dimension  of  a  simplex  is  one  less 
than  the  number  of  its  elements.  If  r  C  a,  with  r,  a  £  S,  then  one  says  that  r  is  a  face  of  a. 

We  permit  the  empty  simplex  0,  for  combinatorial  simplicity  [8,  47].  The  complex  {0}, 
consisting  solely  of  the  empty  simplex,  is  the  empty  complex.  It  is  also  the  sphere  of  dimension 
— 1.  The  complex  0,  consisting  of  no  simplices,  is  the  void  complex. 

If  S  is  a  simplicial  complex,  if  Y!  is  some  subcollection  of  S,  and  if  Y,1  is  a  simplicial  complex 
in  its  own  right,  then  one  says  that  S'  is  a  subcomplex  of  S. 

All  simplicial  complexes  in  this  paper  are  finite.  Any  nonvoid  finite  simplicial  complex 
has  a  geometric  realization  in  some  Euclidean  space,  with  relative  topology  the  same  as  its 
polytope  topology  [61].  Thus  we  may  view  S  as  a  topological  space. 

See  [61,  6,  39]  for  a  further  introduction  to  topology  and  simplicial  complexes.  See  Section 
5  for  a  summary  of  topological  tools  used  in  this  paper. 

Strategy  Complexes 

Given  a  nondeterministic  graph  G  =  {V.  A)  with  F  /  0,  let  A q  be  the  simplicial  complex 
whose  simplices  are  the  acyclic  collections  of  actions  BCd.  If  V  =  0,  let  Aq  =  0.  We  refer 
to  A g  as  G1  s  strategy  complex  and  to  every  simplex  in  Aq  as  a  (nondeterministic)  strategy. 
Observe  that  no  action  of  G  with  a  self- loop  can  appear  in  any  simplex  of  Ac- 

3.2  Examples 

In  what  follows,  we  abbreviate  any  action  {(u,  «i), . . . ,  (v,  u &)}  by  writing  v  — ►  ul5 . . . ,  uk. 

The  graph  on  the  left  of  Fig.  5  is  a  standard  directed  graph.  Each  edge  of  the  directed 
graph  is  a  possible  “action”  the  system  could  perform,  moving  it  from  some  state  of  the  graph 
to  some  other  state. 

The  strategy  complex  of  this  graph  is  shown  in  the  right  panel  of  Fig.  5.  The  biggest  simplex 
one  could  possibly  expect  to  see  in  the  strategy  complex  would  be  a  tetrahedron,  consisting 
of  all  four  actions  present  in  the  directed  graph.  However,  two  of  the  actions,  namely  1  — >  2 
and  2  — >  1,  could  give  rise  to  a  cycle  in  the  graph,  so  no  simplex  of  the  strategy  complex  can 
contain  both  these  actions.  The  complex  is  in  fact  generated  by  two  triangles. 
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This  edge  constitutes 
the  strategy  to  move  away 
from  graph  state  1 . 


2  -  This  triangle 

constitutes  one  strategy 
for  attaining  graph  state  3. 


This  edge  is 
the  backchained 
strategy  for  attaining 
graph  state  3. 


Figure  5:  The  graph  on  the  left  defines  the  strategy  complex  shown  on  the  right. 


The  two  triangles,  as  well  as  three  of  the  five  edges  in  the  complex,  constitute  strategies 
for  attaining  state  3  in  the  graph.  The  central  edge,  consisting  of  the  actions  { 1  — >  3;  2  — >  3}, 
is  the  strategy  one  would  obtain  by  backchaining  from  state  3  in  a  traditional  fashion. 

Observe  that  a  strategy  complex  may  contain  strategies  for  a  variety  of  goals.  For  instance, 
the  top  left  edge  of  the  complex  in  Fig.  5,  comprising  the  set  of  actions  {1  — >  3;  1  — *  2},  is  a 
strategy  that  simply  says  “move  away  from  state  1.” 


Figure  6:  The  graph  on  the  left  has  two  nondeterministic  actions  that  could  create  a  cycle,  so 
the  strategy  complex  on  the  right  consists  of  two  isolated  vertices. 

For  contrast,  consider  the  graph  of  Fig.  6.  It  contains  two  actions,  one  each  at  states  1  and 
2.  Each  action  has  two  nondeterministic  outcomes.  The  two  actions  cannot  appear  together  as 
a  simplex  since,  depending  on  the  actual  nondeterministic  transitions  at  runtime,  these  actions 
could  cause  cycling  in  the  graph  between  states  1  and  2.  As  a  result,  the  strategy  complex 
consists  of  two  isolated  vertices,  representing  the  two  strategies  “move  away  from  state  1”  and 
“move  away  from  state  2.”  In  particular,  there  are  no  strategies  guaranteed  to  attain  state  3 
from  the  other  two  states. 
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1— >2 


2^3 


3^1 


Figure  7:  A  strongly  connected  directed  graph  and  its  strategy  complex.  Compare  with  Fig.  8. 


1— >3  2— >3 


o^^o^=^o 


3— >2  3— >1 

Figure  8:  Another  strongly  connected  directed  graph  and  its  strategy  complex.  See  also  Fig.  7. 

It  is  also  instructive  to  compare  the  two  graphs  and  strategy  complexes  shown  in  Figures 
7  and  8.  Both  graphs  are  strongly  connected  directed  graphs  with  three  states.  The  graphs 
are  not  isomorphic  or  even  homomorphic,  but  their  strategy  complexes  are  both  topological 
circles. 

Indeed,  Hultman  [44]  proved:  Any  directed  graph  that  can  be  written  as  the  disjoint  union 
of  its  strongly  connected  components  generates  a  strategy  complex  (he  used  a  different  name) 
topologically  similar  to  a  sphere  of  dimension  n  —  k  —  1,  where  n  is  the  number  of  states  in 
the  graph  and  k  is  the  number  of  strongly  connected  components.  All  other  directed  graphs 
produce  contractible  strategy  complexes  (roughly  meaning:  they  can  be  shrunk  to  a  point;  see 
Section  5  for  a  precise  definition) . 

This  is  a  first  hint  that  topology  is  capturing  some  significant  graph  property  independent 
of  the  detailed  structure  of  a  graph.  In  the  case  of  directed  graphs,  the  strategy  complexes 
always  look  either  like  spheres  or  contractible  spaces  (homotopically;  see  Section  5). 

Remark:  The  nondeterministic  setting  is  considerably  richer  than  the  deterministic  setting 
of  directed  graphs.  As  we  will  see  in  Section  13,  nondeterministic  graphs  are  able  to  generate 
a  much  larger  collection  of  topological  spaces  via  their  strategy  complexes,  namely  all  spaces 
describable  by  finite  simplicial  complexes.  This  means  nondeterministic  graphs  and  finite 
simplicial  complexes  are  essentially  identical  topological  objects.  Consequently,  one  should  use 
topology  to  study  strategies  for  solving  tasks  in  uncertain  discrete  spaces. 
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4  Loopback  Graphs  and  Complexes 


Now  let  us  modify  the  graph  of  Fig.  5.  We  will  add  artificial  deterministic  transitions  from 
state  3  to  each  of  states  1  and  2.  We  call  these  added  transitions  loopback  actions.  Think  of 
these  loopback  actions  as  “topological  electrodes”  that  will  allow  us  to  measure  whether  the 
graph  contains  a  guaranteed  strategy  for  attaining  state  3  from  all  states. 


Figure  9:  A  loopback  graph  and  loopback  complex  associated  with  the  graph  of  Fig.  5.  The 
complex  contains  6  vertices,  12  edges,  and  6  triangles  (shaded).  The  two  triangular  endcaps 
outlined  in  dashed  red  are  not  part  of  the  complex,  since  each  gives  rise  to  a  cycle  in  the  graph. 
The  complex  is  homotopic  to  S'1,  the  circle. 

The  left  panel  of  Fig.  9  shows  the  resulting  loopback  graph.  Now  imagine  constructing  the 
strategy  complex  associated  with  that  graph,  as  shown  in  the  right  panel  of  the  figure.  We 
refer  to  it  as  a  loopback  complex  of  our  original  graph.  The  complex  in  this  case  looks  roughly 
like  a  polygonal  cylinder.  The  complex  is  homotopic  to  a  circle,  as  represented  by  either  open 
end  of  the  cylinder. 

“Homotopic”  means,  in  this  case,  that  the  complex,  viewed  as  a  topological  space,  can 
be  continuously  deformed  within  itself  into  a  subspace  that  is  topologically  a  circle.  Notice 
that  one  cannot  continuously  deform  the  complex  (within  itself)  into  a  point.  This  is  crucial. 
Homotopy  type  is  an  equivalence  relation  on  topological  spaces.  Circles  and  points  lie  in 
different  homotopy  equivalence  classes.  (See  Section  5  for  precise  definitions.) 


Figure  10:  A  loopback  graph  and  loopback  complex  associated  with  the  graph  of  Fig.  6.  The 
complex  is  contractible. 

In  contrast,  suppose  we  add  both  possible  loopback  actions  at  state  3  to  the  graph  of  Fig.  6. 
The  resulting  graph  and  loopback  complex  are  shown  in  Fig.  10.  Now  the  loopback  complex 
is  homotopic  to  a  point;  one  can  continuously  deform  it  within  itself  to  a  point. 
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The  Punch  Line:  No  matter  how  complicated  the  nondeterministic  graph,  if  we  add  all 
loopback  actions  to  it  that  transition  from  some  state  s  to  the  remaining  states,  then  the 
resulting  loopback  complex  will  always  be  homotopic  either  to  a  sphere  or  to  a  point.  A  sphere 
tells  us  that  there  is  a  strategy  guaranteed  to  attain  state  s  from  all  states  in  the  graph;  a 
point  tells  us  that  no  such  strategy  exists.  We  are  beginning  to  see  a  topological  language  by 
which  to  characterize  system  capabilities. 

The  following  definitions  and  theorem  make  the  previous  observation  precise.  We  provide  a 
proof  of  the  theorem  following  its  statement,  both  to  preserve  continuity  and  to  build  intuition. 
The  reader  may  nonetheless  first  wish  to  review  the  topological  tools  discussed  in  Section  5. 
The  foundations  of  our  proof  appear  in  [8,  44] .  Subsequently,  we  will  generalize  these  techniques 
further,  namely  to  stochastic  graphs. 

Definitions  Let  G  =  (V.  A)  be  a  nondeterministic  graph  and  suppose  s  G  V  is  some  desired 
stop  state.  We  make  the  following  definitions: 

•  G  contains  a  complete  guaranteed  strategy  for  attaining  s  if  there  is  some  acyclic  set  of 
actions  B  C  A  such  that  B  contains  at  least  one  action  with  source  v  for  every  v  €  L\{s}. 

Observe  that  B  cannot  contain  any  actions  with  source  s.  Moreover,  any  possible  path  in 
the  graph  (V,  B),  that  terminates  at  some  Vk  A  s,  may  be  extended  to  a  longer  possible 
path.  Iterating,  this  process  converges  at  s,  since  B  is  acyclic. 

We  say:  B  is  a  complete  guaranteed  strategy  for  attaining  s. 

•  Define  GAS  to  be  the  nondeterministic  graph  identical  to  G  except  that  all  actions  with 
source  s  have  been  discarded,  replaced  instead  by  (|V|  —  l)-many  loopback  actions  {(s,  u)}, 
each  consisting  of  a  single  edge  from  s  to  some  v,  with  v  ranging  over  V\{s}. 

G<-s  is  a  loopback  graph  of  G. 

•  Define  A q^_s  to  be  the  strategy  complex  associated  with  Gh_s. 

is  a  loopback  complex  of  G. 

•  [n]  is  (standard)  shorthand  for  the  set  {1, . . . ,  n}. 

Theorem  1  (Goal  Attainability)  Let  G  =  (V,  A)  be  a  nondeterministic  graph  and  s  €  V . 

If  G  contains  a  complete  guaranteed  strategy  for  attaining  s,  then  A g^s  is  homotopic  to 
the  sphere  Sn~2,  with  n  =  \V\.  Otherwise,  A g^s  is  contractible. 

Proof.  We  may  assume  V  =  [n]  and  s  =  n.  The  theorem  is  trivially  true  for  n  =  1,  so 
suppose  n  >  1. 

I.  Let  B  be  a  complete  guaranteed  strategy  for  attaining  s  and  let  A!  be  the  actions  of 
GAS.  For  each  A  e  A1 ,  define  the  open  polyhedral  cone  Ua  =  f)(i,j)eA{x  £  R”  |  xi  >  xj}. 
Observe  that  a  set  of  actions  {A\, . . . ,  A^}  is  acyclic  if  and  only  if  Ua1  D  ' ' '  D  Ua,.  is  not  empty. 
When  nonempty,  the  intersection  is  contractible.  By  the  Nerve  Lemma  (see  Section  5.3),  A g^s 
therefore  has  the  homotopy  type  of  LUe.4'  Ua-  We  claim  that  this  union  covers  all  of  Rn  except 
for  the  line  on  which  all  coordinates  are  equal.  Thus  it  is  homotopic  to  S”1-2. 

To  see  coverage:  Clearly  no  point  with  all  coordinates  equal  can  be  in  the  union.  The 
cones  determined  by  the  loopback  actions  cover  all  points  x  E  R”  for  which  xn  >  xt,  some  i. 
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Suppose  some  x  in  Rra\{xi  =  •  •  •  =  xn}  is  not  inside  any  U a-  Then  xt  >  xn  for  all  i,  with  at 
least  one  Xi  >  xn.  Some  action  B  £  B  has  that  i  as  a  source.  B  C  A! ,  so  x  0  Ub,  meaning 
there  is  some  target  j  of  B  such  that  Xj  >  Xi  >  xn.  Repeating  this  argument  with  j.  etc., 
produces  an  arbitrarily  long  and  thus  cyclic  possible  path  in  Hg.  Contradiction. 

II.  If  G  does  not  contain  a  complete  guaranteed  strategy  for  attaining  s,  then  no  simplex 
of  A g^s  contains  actions  at  all  states  of  V\{s}.  For  every  simplex  a  £  A g^s  there  is  therefore 
a  unique  nonempty  maximal  set  tct  of  loopback  actions  such  that  a  (J  rCT  £  A •  A  standard 
collapsing  argument  now  shows  that  A g^s  is  contractible  (see  Section  5.2).  □ 


5  Topological  Tools  and  Homotopy  Equivalence 

One  of  the  most  important  functions  of  topology  is  to  recognize  equivalences  between  seemingly 
different  objects.  This  section  reviews  some  of  the  key  topological  equivalences  and  tools 
for  establishing  those  equivalences.  As  suggested  by  Theorem  1,  we  will  use  these  tools  to 
characterize  system  capabilities  by  assigning  topological  structures  to  nondeterministic  (and 
stochastic)  graphs. 

Two  simplicial  complexes,  T  and  S,  are  said  to  be  isomorphic,  written  T  =  S,  if  there  is  a 
bijective  correspondence  between  the  vertices  of  the  two  complexes  given  by  some  function  /, 
such  that  {v\, . . . ,  Vk}  is  a  simplex  of  T  if  and  only  if  {/(tq), . . . ,  f(vk)}  is  a  simplex  of  S.  In 
particular,  /  preserves  simplex  dimension. 

Two  topological  spaces,  X  and  Y,  are  said  to  be  homeomorphic,  written  X  ~  Y,  if  there 
exist  two  continuous  functions,  /  :  X  — ►  Y  and  g  :  Y  <■  X,  such  that  g  o  /  =  idx  and 
/  o  g  =  idy,  where  idx  is  the  identity  function  on  X  and  idy  is  the  identity  function  on  Y. 

Isomorphic  simplicial  complexes  are  homeomorphic  when  viewed  as  topological  spaces. 

~  is  an  equivalence  relation  in  the  category  of  topological  spaces.  It  is  perhaps  the  most 
familiar  to  the  reader.  Two  homeomorphic  spaces  really  are  very  much  the  same  intuitively, 
differing  “only”  by  some  continuous  transformation  with  a  continuous  inverse. 

There  is  a  weaker  equivalence  relation  for  topological  spaces,  based  on  homotopy.  This 
relation  is  more  akin  to  the  notion  of  “morphing”  similarity  one  sees  in  computer  graphics.  The 
equivalence  classes  one  obtains  under  homotopy  turn  out  to  be  useful  descriptors  for  planning 
in  the  presence  of  uncertainty.  We  can  think  of  these  equivalence  classes  as  constituting  major 
nouns  of  the  language  we  desired  in  Section  1.  We  give  the  definitions  next  and  then  some 
tools  for  establishing  homotopy  equivalence. 

Suppose  /o  :  X  — >  Y  and  fi  :  X  — >  Y  are  two  continuous  functions  between  topological 
spaces  X  and  Y.  One  says  that  /o  is  homotopic  to  f\,  written  /o  —  /i,  if  there  exists  a 
continuous  function  F  :  X  x  [0, 1]  — >  Y  such  that  F(x,  0)  =  fo(x)  and  F(x,  1)  =  fi(x)  for  all 
x  £  X.  (Here  [0, 1]  is  the  unit  interval  of  the  real  line.)  One  can  think  of  F(-,t )  as  a  continuous 
“morphing”  of  fo  into  f\  as  t  varies  from  0  to  1. 

Two  topological  spaces,  X  and  Y,  are  said  to  be  homotopy  equivalent  (or  to  have  the  same 
homotopy  type  or  to  be  homotopic),  written  X  ~  Y,  if  there  exist  two  continuous  functions, 
/  :  X  —*  Y  and  g  :  Y  — >  X,  such  that  go/~  idx  and  /  og~  idy.  Observe  that  the  difference 
between  “homeomorphic”  and  “homotopic”  is  the  difference  between  compositions  that  are 
exactly  equal  to  the  identity  versus  merely  homotopic  to  the  identity.  Thus  homeomorphic 
spaces  are  also  homotopic. 
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A  topological  space  homotopic  to  a  single  point  is  said  to  be  contractible. 

It  is  often  difficult  to  establish  homotopy  equivalence  based  on  these  definitions  alone,  but 
there  are  several  key  topological  tools  that  one  may  use  instead.  We  discuss  these  next.  Once 
again,  we  also  refer  the  reader  to  [61,  39,  6,  8]  for  more  extensive  treatments. 

5.1  Deformation  Retractions 

Suppose  A  is  a  subspace  of  a  topological  space  X.  A  (strong)  deformation  retraction  of  X 
onto  A  is  a  continuous  function  F:Ix[0,l]->I  such  that: 

(a)  F(x,  0)  =  x  for  all  i£l, 

(b)  F(x,  1)  G  A  for  all  x  G  X,  and 

(c)  F(a,  t)  =  a  for  all  a  G  A  and  all  t  G  [0, 1]. 

F  establishes  that  X  ~  A  [61]  and  one  says  that  A  is  a  deformation  retract  of  X.  More 
generally,  it  is  a  fact  that  two  spaces  are  homotopy  equivalent  precisely  when  each  may  be 
viewed  as  a  deformation  retract  of  some  common  encompassing  space  [39] . 

We  encourage  the  reader  to  verify  the  following  facts,  as  warmup  for  ideas  to  come: 

(1)  Let  X  be  all  of  n-dimensional  Euclidean  space  except  for  the  origin. 

Then  X  is  homotopy  equivalent  to  S'”-1,  the  sphere  of  dimension  n  —  1. 

(2)  Let  X  =  R”  \  {x  G  R”  |  x\  =  ■  ■  ■  =  xn },  that  is,  n-dimensional  Euclidean 
space  with  a  line  removed.  Then  X  is  homotopy  equivalent  to  S”-2. 

(The  proof  of  Theorem  1  uses  this  fact.) 


5.2  Collapsibility  and  Contractibility 

Suppose  E  is  a  simplicial  complex  and  suppose  r  and  a  are  simplices  in  E  such  that  r  is  a 
proper  face  of  a.  If  r  is  a  proper  face  of  no  other  simplex  in  E,  then  one  can  remove  both  r 
and  <7  from  E  to  obtain  a  new  complex  E'  =  E  \  {r,  a}  that  has  the  same  homotopy  type  as  E. 
The  process  of  constructing  T,'  from  E  is  called  an  elementary  collapse.  The  reverse  process  is 
called  an  elementary  anti- collapse.  [6] 

It  is  a  fact  that  a  finite  simplicial  complex  is  contractible  if  and  only  if  there  is  a  sequence 
of  elementary  collapses  and  elementary  anti-collapses  that  transforms  the  complex  into  a  single 
point.  Finding  such  a  sequence  is  an  uncomputable  problem  (this  goes  back  to  a  deep  result  that 
the  word  problem  for  groups  is  undecidable  [63]).  The  special  case  in  which  only  elementary 
collapses  are  needed  is  computable;  one  can  try  all  possibilities.  Such  complexes  are  called 
collapsible.  A  classic  example  of  a  space  that  is  contractible  but  not  collapsible  is  the  “House 
with  Two  Rooms”  [39]. 

A  finite  simplicial  complex  E  is  a  cone  if  there  is  some  vertex  v  of  E  such  that  a  (J  {u}  G  E 
whenever  a  G  E.  In  this  case,  v  is  an  apex  of  the  cone.  A  cone  is  a  classic  example  of  a 
collapsible  complex. 
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The  following  very  useful  fact  appears  as  Lemma  7.6  in  [8]  (the  collapsibility  argument  in  the 
proof  of  Theorem  1  uses  this  lemma): 

Notation:  a  ±  v  means  a  \  {u}  if  v  £  a  and  a  IJ  {u}  if  v  fL  a. 

Lemma  2  (Bjorner  and  Welker,  [8])  Suppose  S'  is  a  subcomplex  of  a  finite  simplicial 
complex  E  and  suppose  for  some  vertex  v  of  E,  the  collection  E  \  S'  is  closed  under  the  map 
a  i— >  a±v.  Then  E  collapses  to  E'  (via  a  sequence  of  elementary  collapses)  and  thus  E  ~  S'. 

5.3  The  Nerve  Lemma 

Suppose  U  is  some  collection  of  sets  (not  necessarily  distinct).  One  may  define  a  simplicial 
complex  called  the  nerve  of  U,  written  M(U),  as  follows:  The  simplices  of  A f[U)  are  given 
by  the  empty  simplex  and  all  nonvoid  finite  subcollections  {74, . . . ,  74}  of  U  such  that  the 
intersection  74  f)  •  •  •  f)  Uu  is  not  empty.  See  [61,  39]. 

Lemma  3  (Nerve  Lemma,  [39])  Let  X  be  a  paracompact  topological  space.  Suppose  U  is  a 
collection  of  open  subsets  of  X  whose  union  covers  X,  such  that  the  intersection  of  any  nonvoid 
finite  subcollection  of  sets  in  U  is  contractible  whenever  it  is  nonempty.  Then  X  ~  M(U). 

The  reader  can  look  up  the  term  “paracompact”.  See  [50,  20,  39].  It  is  enough  for  our 
purposes  to  know  that  any  topological  subspace  of  Euclidean  space  is  paracompact. 

The  Nerve  Lemma  infers  global  homotopy  type  from  local  contractibility.  For  instance, 
in  Theorem  1,  we  associated  to  every  action  of  a  nondeternrinistic  graph  an  open  set.  The 
Nerve  Lemma  allowed  us  to  infer  the  overall  topology  of  the  graph’s  strategy  complex  by 
the  intersection  properties  of  these  open  sets.  In  Section  7,  we  will  associate  open  sets  with 
actions  in  both  nondeternrinistic  and  stochastic  graphs  so  as  to  encode  the  actions’  convergence 
properties.  Again,  the  Nerve  Lemma  will  allow  us  to  infer  the  topology  of  the  graphs’  strategy 
complexes. 

5.4  The  Quillen  Fiber  Lemma 

Another  very  useful  tool  is  the  Quillen  Fiber  Lemma.  We  will  state  it  for  partially  ordered  sets 
(known  as  posets ).  Every  simplicial  complex  E  defines  a  poset  4(E),  called  the  face  poset  of  E. 
The  elements  of  the  face  poset  4(E)  are  the  nonempty  simplices  of  the  complex  E,  partially 
ordered  by  set  inclusion.  One  can  also  construct  a  simplicial  complex  E (P),  called  the  order 
complex,  from  any  poset  P.  The  simplices  of  E (4)  are  given  by  the  finite  chains  pi  <  ■  ■  ■  <  Pk 
in  P. 

It  is  a  fact  that  E  and  E(4(E))  are  homeomorphic.  Indeed,  E(4(E))  is  the  first  barycentric 
subdivision  of  E,  usually  written  as  sd(E).  Abstractly,  sd(E)  is  a  new  complex,  whose  nonempty 
simplices  are  given  by  all  sets  of  the  form  {<7i,  02,  ■  •  • ,  0fc},  with  each  a  nonempty  simplex 
of  E,  and  with  cr,;  a  proper  face  of  Ui+i,  for  i  =  —  1  [69].  Geometrically,  the  first 

barycentric  subdivision  is  a  re-triangulation  obtained  by  adding  as  vertices  the  centroids  (called 
barycenters )  of  all  simplices  in  E,  then  defining  new  simplices  accordingly  [71,  61]. 

Thus  posets  and  simplicial  complexes  are  essentially  identical  topological  objects.  For 
instance,  one  may  speak  of  the  topology  of  a  poset,  implicitly  meaning  the  topology  of  its 
order  complex.  We  will  not  elaborate  on  this  connection  further,  but  refer  the  reader  to 
[8,  6,  75], 
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Notation:  If  Q  is  a  poset,  then  Q<q  denotes  the  set  {g'  G  Q  |  q'  <q  q},  where  <q  is  the 
partial  order  on  Q  (set  inclusion  in  the  case  of  face  posets  derived  from  simplicial  complexes). 

Lemma  4  (Quillen,  [68])  Suppose  f  :  P  — >  Q  is  an  order-preserving  map  between  two 
posets.  If  f~1(Q<q)  is  contractible  for  all  q  G  Q,  then  P  and  Q  are  homotopy  equivalent. 

5.5  Homotopy  Interpretation 

It  is  still  a  research  question  to  determine  fully  what  the  homotopy  type  of  a  graph’s  strategy 
complex  really  means.  Much  of  the  rest  of  the  paper  will  be  devoted  to  understanding  that 
meaning  for  some  important  special  cases. 

The  encircled  vertex,  and  the  two  edges 
and  triangle  touching  it,  collapse  away. 


Figure  11:  The  deterministic  action  1  — »  2  is  more  precise  than  the  nondeterministic  action 
1  — >  2,3.  The  strategy  complex  makes  this  explicit,  by  showing  how  simplices  containing 
action  1  — >■  2,  3  can  collapse  away  while  preserving  homotopy  type. 

There  is  however  something  very  simple  we  can  observe.  Suppose  G  =  (V,  A)  is  a 
nondeterministic  graph.  Suppose  G  contains  two  distinct  actions  A  €  A  and  B  €  A  with 
identical  source  states  such  that  one  action  is  more  precise  than  the  other.  Viewed  as  edge 
sets,  suppose  AC  B.  Intuitively,  we  would  expect  never  to  need  action  B.  After  all,  anything 
we  can  be  certain  of  doing  with  action  B,  we  can  also  be  certain  of  doing  with  action  A. 

Homotopy  equivalence  captures  this  observation  via  collapsibility.  Figure  Fig.  11  provides 
an  example. 

In  particular,  suppose  B  e  a  £  Aq.  Let  r  =  a  ±  A.  Then  Here  Aq  as  well.  So 
Lemma  2  tells  us  that  the  collection  A-  =  {er  £  Aq  \  B  0  a}  is  homotopy  equivalent  to  Aq. 
In  other  words,  action  B  is  irrelevant.  Observe  as  well  that  A-  =  AG- ,  where  G~  is  the 
nondeterministic  graph  identical  to  G  except  that  action  B  has  been  removed. 

6  Stochastic  Graphs  and  Strategy  Complexes 

6.1  Stochastic  Actions  and  Graphs 
Stochastic  Actions 

Section  3  defined  a  nondeterministic  action  to  be  a  nonempty  set  of  directed  edges  with 
common  source.  We  now  define  a  stochastic  action  to  be  a  nonempty  set  of  directed  edges  with 
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common  source  in  which  each  edge  is  labeled  with  a  transition  probability.  Formally,  we  write 
a  stochastic  action  A  as  a  nonempty  set  of  labeled  pairs  in  the  form  {(v,p\Ui),  (v,p2U2),  ■  ■  •}, 
with  v  and  all  m  elements  of  some  underlying  state  space  V .  As  before,  v  is  A’s  source  and 
each  Ui  is  a  (stochastic)  target  of  A.  Each  label  pi  is  a  transition  probability. 

The  semantics  of  a  stochastic  action  are  Markovian:  Action  A  may  be  executed  whenever 
the  system  is  at  state  v.  When  action  A  is  executed,  the  system  moves  from  state  v  to  one  of 
the  targets  iq,  selected  from  all  of  A’s  targets  with  probability  jp.  We  require  that  each  pi  >  0 
and  that  YhiVi  =  1- 


Stochastic  Graphs 

A  stochastic  graph  G  =  (V,A)  is  a  set  of  states  V  and  a  collection  of  actions  A  whose 
sources  and  targets  all  lie  in  V.  V  is  also  known  as  G’ s  state  space.  An  action  may  be 
either  nondeterministic  or  stochastic.  As  before,  distinct  actions  may  have  overlapping  or 
identical  edge  sets  or  even  identical  sets  of  transition  probabilities.  And,  again,  all  graphs,  sets 
of  states,  actions,  and  collections  of  actions  in  this  paper  are  finite. 

A  stochastic  subgraph  H  =  (W,B)  of  a  stochastic  graph  G  =  (V,A)  is  a  stochastic  graph 
in  its  own  right  such  that  W  C  V  and  B  C  A. 

Remarks: 

•  To  emphasize:  We  allow  both  nondeterministic  and  stochastic  actions  in  a  stochastic 
graph. 

•  Determinism:  We  may  view  a  deterministic  action  that  transitions  from  state  v  to  state 
u  as  a  special  case  of  either  a  nondeterministic  action,  {(i;,  it)},  or  a  stochastic  action, 
{(u,  lit)}.  In  this  paper,  it  does  not  matter  which. 

•  Notation:  In  figures  of  graphs,  we  label  the  edges  of  a  stochastic  action  with  transition 
probabilities.  In  figures  of  strategy  complexes,  we  indicate  the  vertex  corresponding  to 
a  stochastic  action  A  =  {(v,piui), . . . ,  (v,pkUk)}  with  the  label  v  — >  piui, . . . ,  pkuk.  We 
sometimes  use  this  notation  in  the  text  as  well. 

6.2  Stochastic  Acyclicity 


Figure  12:  The  graph  on  the  left  has  two  stochastic  actions  that  together  could  create  a  cycle 
but  must  eventually  converge  to  state  3.  Consequently,  the  strategy  complex  on  the  right 
includes  not  only  the  individual  actions  but  the  full  1-simp  lex  formed  by  these  two  actions. 


21 


The  two  nondeterministic  actions  of  the  graph  in  Fig.  6  did  not  appear  together  in  a  strategy 
simplex  since  together  they  could  lead  to  infinite  looping.  Now  imagine  that  the  actions  are 
merely  stochastic,  as  in  Fig.  12.  Intuitively,  we  probably  do  want  to  permit  both  these  actions 
together  in  a  strategy  simplex.  Although  such  a  strategy  might  lead  to  cycling  between  states 
1  and  2,  the  cycling  would  be  stochastic  and  therefore  not  last  forever.  Eventually  the  system 
would  converge  to  state  3.  Indeed,  one  may  even  compute  the  expected  convergence  times. 

We  therefore  need  to  modify  our  definition  of  acyclicity  to  account  for  stochastic 
convergence.  There  are  two  natural  approaches,  both  grounded  in  the  idea  that  an  adversary 
might  try  to  choose  actions  and  transitions  in  such  a  way  as  to  keep  the  system  cycling  for  as 
long  as  possible. 

Definition:  Given  a  collection  of  actions  B,  let  Vs  =  {u  |  v  is  the  source  of  some  BgB}. 
We  refer  to  Vs  as  the  start  region  of  B. 


Markov  Chain  Perspective:  Suppose  G  =  (V,A)  is  a  stochastic  graph  (so  it  may  contain 
either  nondeterministic  or  stochastic  actions  or  both).  Let  IF  be  some  subset  of  VA-  Now 
imagine  an  adversary  who  constructs  a  Markov  chain  M  as  follows: 

•  For  every  v  £  IF,  the  adversary  selects  some  action  A  £  A  whose  source  is  v. 

—  If  the  action  A  is  stochastic,  that  is,  A  =  {(v , piu±) , . . . ,  (v , pk'Uk)} ,  then  the 
stochastic  transitions  of  M  at  v  are  exactly  those  given  by  A. 

—  If  the  action  A  is  nondeterministic,  that  is,  A  =  {(u,  u±), . . . ,  (v,  Uk)j,  then  the 
adversary  further  selects  one  target  Ui  of  A.  There  will  be  exactly  one  transition  of 
M  at  v,  given  by  an  edge  from  v  to  ut .  occurring  with  probability  1. 

•  For  every  v  £  V  \  IF,  there  is  a  single  transition  of  M  at  v,  consisting  of  a  self-loop  with 
probability  1. 

We  refer  to  such  a  construction  as  a  Markov  chain  M  with  support  IF  and  say  G  contains 
(M,  IF).  Observe  that  in  general  G  may  contain  many  different  (M,  IF). 


Convergence  Time  Perspective:  Suppose  G  =  (F,  A)  is  a  stochastic  graph.  Associate 
to  every  action  A  €  A  a  nonnegative  transition  time  5a-  Consider  the  following  system  of 
equations: 


t 


V 


t 


V 


( 


\ 


max 


max 

A  £  A 


|max  tUj  +  5a 


y  A  =  {( v,uj )} 


A 


max  £  Pjt-Uj  +  5  a 

A  £  A  \  3 

{(v,P]Uj)} 


0,  if  v  ^  VA. 


if  v  £  F4; 


(1) 
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The  reader  may  recognize  System  (1)  as  describing  a  maximization  over  expected  durations 
of  random  walks  with  variable  step  times  [32] ,  induced  by  Markov  chains  (M,  V4)  contained 
in  G.  It  is  a  form  of  Bellman’s  equation,  representing  an  optimization  from  an  imagined 
adversary’s  perspective,  who  is  trying  to  maximize  the  graph’s  convergence  times. 

The  convergence  times  are  0  for  all  states  at  which  there  are  no  actions.  Otherwise,  the 
maximizations  appearing  in  System  (1)  describe  adversarial  choices.  At  any  state  v,  the 
adversary  maximizes  over  all  actions  A  with  source  v.  If  A  is  a  nondeterministic  action, 
the  adversary  performs  an  additional  maximization  over  the  convergence  times  of  all  targets 
of  A;  if  A  is  stochastic,  there  appears  an  expectation  over  the  convergence  times  of  the  target 
states. 

Reminder:  A  recurrent  class  of  a  Markov  chain  M  is  a  set  of  states  R  of  M  such  that  the 
probability  of  reaching  any  state  of  R  from  any  other  state  of  R  is  1,  while  the  probability  of 
reaching  any  state  outside  R  is  0.  The  restriction  of  M  to  R  induced  by  transitions  at  states  of 
R  therefore  itself  defines  a  Markov  chain.  If  R  consists  of  a  single  state,  that  state  is  called 
absorbing.  [32,  48] 

The  following  lemma  establishes  an  equivalence  between  the  Markov  chain  perspective  and 
the  convergence  time  perspective: 

Lemma  5  (Stochastic  Acyclicity)  Let  G  =  (V,A)  be  a  stochastic  graph  with  associated 
nonnegative  action  transition  times  {5a}a&a- 

System  (1)  has  a  unique  finite  solution  if  and  only  if  the  only  recurrent  classes  of  any 
Markov  chain  with  support  W  contained  in  G  are  formed  by  the  absorbing  states  V  \  W . 

Moreover,  when  the  solution  is  unique  and  finite,  it  is  nonnegative,  that  is,  tv  >  0  for  all 
veV. 

Proof.  We  omit  the  details  of  the  proof.  The  basic  techniques  are  similar  to  those  used  in 
Markov  Decision  Processes.  We  point  to  [32,  48,  78].  □ 

The  previous  lemma  permits  us  to  move  back  and  forth  between  the  Markov  chain  and 
convergence  time  perspectives.  Often  it  is  best  to  reason  directly  about  the  recurrent  classes 
induced  by  a  stochastic  graph  but  easier  to  compute  convergence  times  using  System  (1).  In 
particular,  when  System  (1)  has  a  unique  finite  solution  one  can  use  the  system  in  an  iterative 
fashion  to  obtain  that  solution. 

6.3  Stochastic  Strategy  Complexes 

We  now  make  some  further  definitions,  leading  to  strategy  complexes  in  the  stochastic  setting. 

Stochastically  Acyclic  Collections  of  Actions 

Suppose  G  =  (V,  A)  is  a  stochastic  graph.  G  is  stochastically  acyclic  if  System  (1)  has  a  unique 
finite  solution  for  some  (and  thus  any)  set  of  nonnegative  action  transition  times  {Sa}agA- 
Similarly,  we  say  that  a  collection  of  actions  B  C  A  is  stochastically  acyclic  if  the  induced 
subgraph  Hg  =  (V,B)  is  stochastically  acyclic. 
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When  B  is  stochastically  acyclic,  we  may  view  B  as  a  strategy  for  moving  the  system  into 
the  complement  of  £>'s  start  region  Vg:  If  the  initial  state  of  the  system  lies  inside  Vq,  then 
moving  under  actions  of  B,  the  system  will  eventually  stop  at  some  state  inside  V  \Vq.  If 
the  system  initially  starts  in  V\Vjg,  then  it  remains  there.  Given  specific  nonnegative  action 
transition  times  {6b}bgB,  we  refer  to  the  solution  of  System  (1),  written  out  for  with  those 
transition  times,  as  the  worst-case  expected  convergence  times  of  B. 

Stochastic  Strategy  Complexes 

Given  a  stochastic  graph  G  =  (V,  A)  with  V  ^  0,  let  be  the  simplicial  complex  whose 
simplices  are  the  stochastically  acyclic  collections  of  actions  B  C  A.  If  V  =  0,  let  Aq  =  0. 

This  definition  is  identical  to  the  one  we  gave  earlier  for  nondeterministic  graphs,  but  our 
notion  of  “acyclic  collections  of  actions”  has  now  been  enlarged  to  include  stochastic  actions. 
Again,  we  refer  to  as  G’s  strategy  complex  and  to  every  simplex  in  Aq  as  a  (stochastic) 
strategy. 

Time-Bounded  Strategy  Complexes 

System  (1)  allows  us  to  define  a  tower  of  strategy  complexes  for  any  graph  G  =  (V.A). 

Suppose  we  associate  nonnegative  transition  times  {<5a}as.4  to  the  actions  of  G.  To  every 
a  G  A q  we  can  then  associate  a  maximal  worst-case  expected  convergence  time ,  tmaxCo'),  defined 
to  be  the  maximum  time  tv  obtained  as  a  solution  to  System  (1)  when  written  out  for  the  graph 

(V». 

Let  T  >  0  be  given.  Define  Aq  =  {a  €  A q  \  tmax(a)  <T}.  Then  Aq  is  a  subcomplex  of 
Ac,  representing  all  strategies  whose  maximal  worst-case  expected  convergence  times  are  no 
greater  than  T.  (Exercise  for  the  reader:  Prove  that  removing  actions  from  a  simplex  a  cannot 
raise  tmax(o'))  as  is  required  to  infer  that  Aq  is  a  simplicial  complex.) 
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7  Covering  Sets 


We  wish  to  understand  the  topology  of  strategy  complexes  and  thus  the  capabilities  of 
nondeterministic  and  stochastic  graphs.  Let  us  adopt  a  technique  that  appeared  in  the  proof 
of  Theorem  1  and  associate  to  each  action  an  open  set.  The  topology  of  the  resulting  collection 
of  open  sets  will  be  the  topology  of  a  graph’s  strategy  complex. 

7.1  Homogeneous  Covering  Sets 

Definition:  Suppose  G  =  (V,  .4)  is  a  stochastic  graph  with  V  A  0.  We  can  assume  for 
simplicity  that  V  =  [n],  with  n  >  1 .  We  associate  to  each  A  £  A  an  homogeneous  open  subset 
U a  of  Rn,  which  we  refer  to  as  a  covering  set: 

•  If  the  action  A  is  stochastic,  that  is,  A  =  {(i,Pjj)}  for  some  set  of  targets  {j},  then 


UA 


<  X  G  Rn 

w  1 

Xi  >  l^PjXj 

{ 

0  J 

•  If  the  action  A  is  nondeterministic,  that  is,  A  =  {(*,  j)}  for  some  set  of  targets  {j},  then 


Ua  =  P|  {x  G  Rn  |  Xi  >  Xj  } . 
j 

In  the  stochastic  case,  the  open  set  Ua  is  a  halfspace  of  Rn  whose  defining  hyperplane  has 
a  normal  determined  by  action  As  transition  probabilities.  In  the  nondeterministic  case,  the 
open  set  Ua  is  the  intersection  of  several  such  halfspaces,  one  for  each  possible  nondeterministic 
target.  Observe  that  the  defining  hyperplanes  all  contain  the  line  {x  G  Rn  |  x\  =  •  •  •  =  xn}. 

Remark:  The  collection  of  all  hyperplanes  of  the  form  {x  G  Rn  |  xt  =  Xj }  is  known 
classically  as  the  Type  A  braid  arrangement  in  Rn  [75];  it  is  very  useful  for  studying  the 
poset  of  all  posets  on  n  items  [8].  It  should  therefore  come  as  no  surprise  that  the  open  sets 
{Ua}agA  will  allow  us  to  infer  the  topology  of  G”s  strategy  complex.  In  particular,  for  a 
nondeterministic  graph  G,  we  are  simply  looking  at  a  subposet  of  that  overall  poset  of  posets 
on  n  items.  For  a  stochastic  graph  G,  we  have  effectively  created  a  poset  of  stochastic  partial 
orders,  each  of  which  we  may  think  of  as  a  collection  of  Markov  chains  all  of  whose  states  are 
either  transient  or  trivially  absorbing  (that  was  the  gist  of  Lemma  5). 


7.2  Affine  Covering  Sets 

Definition:  In  order  to  study  the  topology  of  time-bounded  strategy  complexes,  it  is  useful 
to  define  affine  covering  sets.  Suppose  we  associate  nonnegative  transition  times  to 

the  actions  of  G.  Then  we  can  define  for  each  A  G  A  an  affine  open  subset  U\  of  Rn: 


If  A  =  {( i,Pjj )}  is  stochastic,  then  U\  =  jx  G  R"  xi  >  Pjxj  +  Sa  } 


•  If  A  =  {(i,  j)}  is  nondeterministic,  then  U\  =  fi/  {x  £  |  Xi  >  x j  +  6a } ■ 
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7.3  Inferring  Topology  from  Covering  Sets 

The  following  lemma  shows  that  Ac  and  A f  {{Ua}a^a)  are  isomorphic  simplicial  complexes 
(and  therefore  homeomorphic  and  homotopic).  The  lemma  generalizes  to  stochastic  graphs  a 
statement  that  appeared  for  nondeterministic  graphs  early  in  the  proof  of  Theorem  1. 

Lemma  6  (Homogeneous  Nerve)  Let  G  =  (V,A)  be  a  stochastic  graph  with  V  =  [n], 
n  >  0.  Suppose  0  /  B  C  A.  Then: 

P|  Ub  /  0  if  and  only  if  B  is  stochastically  acyclic. 

B&B 

Proof.  Recall  that  Hq  means  the  subgraph  ( [V. ,  B)  of  G. 

I.  Suppose  f| eeB  ub  /  0- 

Choose  x*  €  Rn  so  that  x*  €  Ub  for  all  B  G  B.  Define  5b  for  each  B  G  B  as  follows: 

If  B  =  {( i,Pjj )}  is  stochastic,  let  5b  =  x*  —  JfjPjX*. 

If  B  =  {(i,  j)}  is  nondeterministic,  let  5b  =  x*  —  rna Xj(x*). 

Observe  that  each  5b  >  0. 

Now  consider  the  system  of  equations: 


By  construction,  System  (2)  has  at  least  one  finite  solution,  given  by  x*,  that  is,  tt  =  x*, 
for  all  i  €  V.  (In  fact,  the  solution  holds  for  all  actions  B  £  B,  that  is,  the  maximum  at  state 
i  in  System  (2)  occurs  for  every  action  of  B  with  source  i.) 

Now  suppose  that  Hg  contains  a  Markov  chain  (M,  W)  whose  support  IT  is  a  recurrent 
class.  Let  P  =  (pij)  be  the  probability  transition  matrix  of  M  restricted  to  IT.  P  is  a 
stochastic  matrix  in  its  own  right  since  IT  is  a  recurrent  class.  The  transition  probabilities  pij 
are  determined  from  actions  of  B  via  the  process  outlined  on  page  22.  In  particular,  for  every 
i  G  IT,  some  action  B  G  B  gives  rise  to  M’s  transitions  at  state  i.  Define  5{  =  5b- 
Combining  this  construction  with  System  (2),  we  see  that: 

x*  >  ^  PijX*~  +  5i,  for  all  i  G  IT. 
jew 
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That  is  only  possible  if  5i  <  0  for  some  i  E  W,  since  P  is  a  stochastic  matrix.  Contradiction. 
So,  by  Lemma  5,  B  must  be  stochastically  acyclic. 


II.  Suppose  B  is  stochastically  acyclic. 

For  each  B  E  B,  let  SB  >  0  be  arbitrary.  Now  write  out  System  (1)  for  H B: 


tj  =  max 


max 
B  E  B 


^max  tj  +  5B^j 


max 
BeB 
B  =  {( i,pjj )} 


Pjtj  +  Sb 


if  i  E  VB] 


U  =  0,  if  %<£  VB. 

By  assumption,  this  system  has  a  unique  finite  solution,  call  it  t*. 
Now  consider  B  E  B: 


If  B  =  {( i,Pjj )}  is  stochastic,  then  t*  >  J2j  Pjtj  +  dB  >  V,  Pjl'j- 
If  B  =  is  nondeterministic,  then  t*  >  maxj(t*)  +  5B  >  rriax^ (t* ) . 

So  t*  E  UB  for  all  B  E  B,  establishing  that  f] b&b  Bb  /  0-  □ 


In  order  to  obtain  the  topology  of  time-bounded  strategy  complexes  we  need  to  be  a  little 
more  careful.  There  may  exist  gaps  in  the  affine  covering  sets  near  the  line  on  which  all 
coordinates  are  equal.  We  will  therefore  intersect  the  covering  sets  with  the  boundary  of  a 
polyhedral  cylinder  designed  to  measure  convergence  times. 


Definition:  Given  real  r  >  0,  let  Cr  =  jx  E  R"  | x%  —  Xj\  <  r  for  all  i ,  j  E 
be  the  boundary  of  Cr. 

Observe:  8Cr  is  a  polyhedral  cylinder  with  axis  given  by  the  line  {x  E  Rn 
and  with  r  the  cylinder’s  “radius” . 


[n]  |  and  let  dCr 
|  xi  =  ■  ■  ■  =  xn  } 


Lemma  7  (Affine  Nerve)  Let  G  =  (V,  A)  be  a  stochastic  graph  with  associated  nonnegative 
action  transition  times  {6a}agA-  Assume  V  =  [n]  and  n  >  0.  Let  T  >  0  be  given. 

There  exists  >  0,  such  that  for  every  0  <  e  <  ex'- 

For  every  0  /  B  C  A,  B  is  stochastically  acyclic  with  tmax(B)  <T  if  and  only  if 

n  bb  n  dcT+e  +  0. 

B&B 

We  omit  the  proof,  except  to  note  that  the  existence  of  in  the  proof  depends  on  finiteness 
of  G.  We  now  move  directly  to  the  key  theorem  that  describes  the  topology  of  strategy 
complexes  in  terms  of  the  topology  of  covering  sets  in  Rn: 
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Theorem  8  (Cover  Homotopy)  Let  G  =  (V,A)  be  a  stochastic  graph  with  V  =  [n],  n>  0. 

Then  Aq  —  [J  Ua- 

AeA 

Let  the  action  transition  times  of  G  be  given  by  nonnegative  numbers  {Sa}agA  and  let  T  >  0. 

Then  Atg  ~  |J  U+  f)  dCT+e, 

AeA 

with  0  <  e  <  ct  and  ct  given  as  per  Lemma  7. 

Proof.  First,  observe  that  whenever  a  set  of  the  form  Ua1  D  ' ' '  fl  U Ak  is  nonempty,  then 
it  is  convex  hence  contractible.  We  therefore  obtain 

Ac  =  M({UA}ASA)  ~  U  UA. 

A&A 

The  isomorphism  =  follows  from  Lemma  6  and  the  homotopy  equivalence  ~  follows  from 
the  Nerve  Lemma. 

Second,  observe  that  whenever  a  set  of  the  form  Ua\  fl ' ' '  fl  Uah  fl  dCx+e  is  nonempty, 

then,  while  it  may  not  be  convex,  it  is  the  deformation  retract  of  a  convex  set,  so  is  contractible. 

Consequently,  we  may  apply  Lemma  7  and  the  Nerve  Lemma  to  conclude 

AS  =  =  U  ua  fl  8°r+«.  □ 

^  '  Acz  A 


1^  p2  ,  q3  2— >  p'l ,  q'3 


3 — ^1 , 2 

Figure  13:  The  graph  on  the  left  has  two  stochastic  actions  (at  states  1  and  2)  and  one 
nondeterministic  action  (at  state  3).  Its  strategy  complex,  shown  on  the  right,  is  the  boundary 
of  a  triangle. 

As  an  example,  let  G  be  the  graph  of  Fig.  13.  The  graph  has  three  states  and  three  actions, 
one  at  each  state.  Two  of  the  actions  are  stochastic  and  one  is  nondeterministic.  A q  is  the 
boundary  complex  of  a  triangle. 

Fig.  14  shows  the  previous  lemmas  and  theorem  in  action,  depicting  the  covering  sets 
{U±,  Uff ,  U^}  associated  with  the  three  actions  of  G ,  assuming  all  action  transition  times 
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Figure  14:  This  figure  shows  a  two-dimensional  slice  of  the  covering  sets  associated  with  the 
actions  of  the  graph  in  Fig.  13,  along  with  intersections  of  these  covering  sets.  The  slice 
describes  the  (x\,  X2)-shape  of  the  covering  sets  at  X3  =  1.  The  figure  also  shows  dCr  (drawn 
in  yellow)  for  various  critical  radii  r  (roughly),  with  the  critical  events  circled.  For  reference, 
the  graph’s  strategy  complex  appears  in  the  upper  left  corner. 
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are  1  and  assuming  p'  >  p  >  The  figure  actually  shows  the  (x\,  X2)-slice  of  the  cover 
{U(f  ,  Uff ,  U%}  at  23  =  1.  Since  the  covering  sets  are  invariant  with  respect  to  a  translation 
along  the  direction  (1,1,1),  it  is  enough  to  look  at  such  two-dimensional  slices  to  determine 
the  topologies  of  Aq  and  A q. 

The  covering  set  associated  with  the  action  3  — >  1 ,  2  is  the  intersection  of  two  halfspaces 
in  R3.  It  therefore  appears  as  the  quadrant  {(21,22)  |  x\  <  0  and  22  <  0}  in  Fig.  14.  U+  and 
U2  each  arise  from  stochastic  actions  and  thus  appear  as  affine  halfspaces  in  Fig.  14. 

The  boundary  of  the  cylinder  Cr  is  drawn  for  several  critical  radii,  namely  those  at  which 
some  simplex  of  Ac  appears.  The  innermost  cylinder  has  radius  r  =  1  +  e.  Since  each  action 
has  transition  time  1,  this  cylinder  just  touches  each  of  the  covering  sets  U+,  Uff ,  and  U^. 
From  a  nerve  perspective,  this  geometry  produces  the  three  vertices  of  the  complex  Aq-  As 
the  radius  grows,  the  other  simplices  of  Ac  appear.  The  correspondence  between  the  covering 
geometry  and  the  complex  topology  is  highlighted  with  a  dashed  double  arrow  for  one  critical 
event:  The  intersection  U(f  (fUf(  in  the  cover  {U(~,  C/^,  U^}  corresponds  to  the  1-simplex 
{l  — >  p2,  q3  ;  3  — >  1,  2}  in  the  complex  Aq. 

8  Controllability  of  Motions  in  Stochastic  Graphs 

Theorem  1  provides  a  topological  test  for  guaranteed  goal  reachability  in  nondeterministic 
graphs.  (A  similar  result  holds  in  the  stochastic  setting.)  We  seek  a  more  general  result,  along 
the  lines  of  Hultman’s  topological  characterization  of  strong  connectivity  in  directed  graphs. 
We  have  the  tool  of  covering  sets  from  Section  7,  so  let  us  use  that  tool  to  extract  topology 
from  strategies.  First,  here  are  some 

Definitions:  Throughout,  let  G  =  (V,  A)  be  a  stochastic  graph  and  let  S  be  a  nonempty 

subset  of  V  ( S  stands  for  “stop  states”). 

•  G  is  a  stochastic  strategy  for  attaining  S  if: 

(i)  G  is  stochastically  acyclic,  and 

(ii)  V\VA  C  S  C  V. 

This  definition  ensures  that  G  contains  actions  at  all  states  outside  of  S  and  that  motions 
under  those  actions  eventually  converge  (in  a  subset  of  S). 

Observe  that  VA  cannot  be  all  of  V,  since  G  is  stochastically  acyclic. 

•  G  contains  a  complete  stochastic  strategy  for  attaining  S  (on  the  state  space  V)  if  there 
is  some  set  of  actions  B  C  A  such  that  Hg  =  {V,  B)  is  a  stochastic  strategy  for  attaining 
S.  (This  definition  is  consistent  with  the  definition  of  complete  guaranteed  strategy  from 
Section  4,  p.16.) 

In  this  case:  —  We  refer  to  both  B  and  Hq  as  being  a  complete  stochastic  strategy 

for  attaining  S  (in  the  graph  G  or  on  the  state  space  V ). 

—  We  say  S  is  a  stochastically  attainable  goal  (in  G  or  within  V ). 
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•  Suppose  X  is  a  subset  of  V  [X  stands  for  “initial  states”).  G  contains  a  stochastic  strategy 
for  attaining  S  from  X  if  G  contains  a  subgraph  H  =  (W,B)  such  that  X[fS  C  W  and 
H  is  a  stochastic  strategy  for  attaining  S.  We  refer  to  if  as  a  stochastic  strategy  for 
attaining  S  from  X.  Of  course,  we  may  also  view  H  as  a  complete  stochastic  strategy  for 
attaining  S,  now  on  the  state  space  W. 

This  definition  captures  the  idea  that  a  strategy  for  attaining  some  set  of  states  from 
some  other  set  of  states  may  require  moving  through  some  intermediate  states,  but  not 
necessarily  all  of  V. 

We  will  presently  focus  on  cases  in  which  X  and  S  are  both  singleton  sets.  In  such  cases, 
we  speak  of  attaining  one  state  from  another. 

8.1  Connectivity:  Covers  and  Chains 

The  following  two  lemmas  capture  the  idea  that  stochastically  certain  connectivity  between 
two  states  in  a  stochastic  (or  nondeterministic)  graph  implies  coverage  of  a  particular  halfspace 
in  Rn  by  the  graph’s  covering  sets.  This  fact  appeared  as  a  special  case  in  the  proof  of  Theorem 
1.  Subsequently,  we  will  use  these  two  lemmas  to  characterize  graph  controllability  (Theorems 
11  and  12). 

Lemma  9  (Stochastic  Connectivity)  Let  G  =  (V,  A)  be  a  stochastic  graph  with  V  =  [n], 
n  >  0.  Let  £,  k  E  V. 

Suppose  G  contains  a  stochastic  strategy  for  attaining  state  k  from  state  £.  Then 

{x  E  Rn  \  xp  >  Xk}  C  (J  UA. 

AeA 

Proof.  While  we  could  give  a  direct  proof,  we  may  also  view  this  lemma  as  a  special  case 
of  Lemma  10,  with  all  action  transition  times  zero.  That  lemma  appears  next.  □ 

Lemma  10  (Time-Bounded  Stochastic  Connectivity)  Let  G  =  ( V ,  A)  be  a  stochastic 
graph  with  associated  nonnegative  action  transition  times  {<5a}as.4- 
Assume  V  =  [n]  and  n  >  0.  Let  £ ,  k  E  V  be  given. 

Suppose  G  contains  a  stochastic  strategy  for  attaining  state  k  from  state  £.  Let  tp  be  the 
worst-case  expected  convergence  time  to  attain  k  from  i,  as  given  by  the  solution  of  System  (1) 
when  written  out  for  this  stochastic  strategy.  Then 

{x  e  Rn  |  xe>  xk  +  te}  c  (J  U\. 

AeA 

Proof.  Let  H  =  (W,B)  be  the  subgraph  of  G  constituting  the  given  stochastic  strategy  for 
attaining  k  from  £.  So  £,  k  £  W .  We  can  assume  without  loss  of  generality  that  W  =  [k].  so  it 
is  enough  to  show  that 


jx  £  Rfc  |  X£  >  xk  T  te  }  C  (J  C/+, 

BeB 


with  each  set  Ug  now  a  subset  of  Rfc. 


31 


If  t  =  k,  there  is  nothing  to  prove,  since  the  set  on  the  left  is  the  empty  set,  which  is  a 
subset  of  all  sets.  So  assume  l^k. 

Here  is  System  (1)  written  out  for  H: 


h  =  max 


max 
B  £  B 


^max  tj  +  6 , 


max  l£  Pj  tj  ~\ ~  &B 

BeB  V  j 

B  =  {( i,pjj )} 


,  for  1  <  i  <  k] 


(3) 


4  =  0. 


Since  H  is  stochastically  acyclic,  this  system  has  a  unique  finite  solution  with  all  tj,  >  0. 

Suppose  there  is  some  x*  £  such  that  x\  >  x\  +  4  but  x*  0  Usee  ^b-  Since  the  sets 
Ug  are  invariant  with  respect  to  translation  along  the  line  jx  £  Rfc  |  x\  =  ■  ■  ■  =  j,  we  can 
assume  without  loss  of  generality  that  x*k  =  0. 


Now  define  a  Markov  chain  M  contained  in  H,  with  support  [fc  — 1]: 
•  At  state  k,  let  M  remain  at  k  with  probability  1. 


•  At  state  i  £  [&  — 1],  pick  some  action  B  £  B  whose  source  is  i.  Such  an  action  must 
exist,  since  H  is  a  complete  stochastic  strategy  for  attaining  k  on  the  state  space  W.  Let 
Sj  =  5b- 


—  If  A?  is  stochastic,  that  is,  B  =  {(i,Pjj)}  for  some  set  of  targets  {j},  then  let  M’s 
transitions  at  i  be  exactly  those  of  B. 

Observe  that  x*  <  J2jPjx*j  +  5j,  since  x*  0  U^. 

—  If  B  is  nondeterministic,  then  there  must  be  some  target  j  of  B  such  that  x*  <  xt+5j, 
again  since  x*  0  Ug.  Let  M  move  from  i  to  j  with  probability  1. 

Let  P  =  ( pij )  be  the  probability  transition  matrix  of  M.  (It  is  a  k  x  k  matrix.)  Since  H  is 
stochastically  acyclic,  M  cannot  have  any  recurrent  classes  other  than  the  absorbing  state  k, 
and  therefore  the  following  system  has  a  unique  finite  solution  (this  reasoning  is  at  the  heart 


of  Lemma  5;  see  also  [48]): 

k 

Xj  =  ^  ]  Pij  Xj  -j-  5j , 

for  1  <i  <k\ 

3= 1 

(4) 

?r 

II 

O 

Moreover,  one  may  obtain  the  solution  to  System  (4)  by  iteration,  starting  from  any  initial 
seed  for  aq, . . . ,  Xk-\-  (This  is  a  standard  result  from  Markov  chains;  it  follows  in  particular 
from  the  theorem  on  p.  389  of  [32].) 


We  will  now  iterate  from  two  different  seeds,  obtaining  contradictory  results: 
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Iteration  Scheme  #1: 

•  Initialize  x\°^  =  ti ,  for  i  =  1, . . . ,  k,  where  {t?;}f=i  is  the  solution  to  System  (3). 

•  For  m  =  0, 1, . . iterate  using  the  update  rules: 

x-m+1)  =  X  l);.i for  1  <  *  <  k\ 

3= 1 

xlm+1)  =  0. 

We  claim  that  <  f*  for  all  i  =  1, . . . ,  k  and  all  m  =  0, 1, . . .  . 

To  see  this: 

The  claim  is  certainly  true  for  all  i  when  m  =  0  and  also  for  i  =  k  for  all  m. 
Inductively,  suppose  the  claim  holds  for  some  m  >  0.  Then,  with  1  <  i  <  k: 

x(m+1)  <  Y^,Pijtj  +  8i  <  U- 

3  =  1 

The  first  inequality  follows  from  the  inductive  hypothesis,  the  second  from 
the  fact  that  the  right  side  of  System  (4)  is  a  special  case  appearing  in  the 
maximizations  of  System  (3). 

Consequently,  =  lim  <  tt,  for  i  =  1, . . . ,  k, 

m— >oo  1 

where  {xi}f=1  is  the  solution  to  System  (4). 

Iteration  Scheme  #2: 

•  Initialize  =  x*,  for  i  =  1, . . . ,  k,  where  x*  is  as  supposed  earlier. 

•  For  m  =  0, 1, . . .,  iterate  using  the  update  rules: 

k 

=  X  I'riHj''  +  for  1  <  *  < 

3=1 
=  0. 

We  claim  that  >  x*  for  all  i  =  1, . . . ,  k  and  all  m  =  0, 1, . . .  . 

Verifying: 

Again,  the  claim  is  true  for  all  i  when  m  =  0  and  also  for  i  =  k  for  all  m. 
Inductively,  suppose  the  claim  holds  for  some  m  >  0.  Then,  with  1  <  i  <  k: 

k 

Vi  >  2^Pijxj+0i  >  x{. 

3= 1 

The  first  inequality  follows  from  the  inductive  hypothesis,  the  second  from 
the  construction  of  M. 

Consequently,  this  time  we  see  that 

Xi  =  lim  yjm)  >  x*,  for  i  =  1, . . . ,  k. 
m-^  oo  1 


(m+1) 

Vi 


(m+1) 

Vk 
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Combining  the  results  of  the  two  iteration  schemes,  we  infer  that  x*  <  ti  for  i  =  1, . . . ,  k. 

In  particular,  for  i  =  £,  recalling  the  definition  of  x*,  we  see  that 

tg  =  0  +  ti  =  x*k  +  ti  <  x}  <  ti . 

That  says  tg  <  ti,  a  contradiction.  □ 

8.2  Characterizing  Controllability  with  Spheres 

The  following  two  theorems  characterize  topologically  the  ability  of  a  finite  discrete  system  to 
reach  any  state  from  any  other  state  despite  control  uncertainty. 

Theorem  11  (Graph  Controllability)  Let  G  =  (V,  A)  be  a  stochastic  graph  with  V  ^  0. 
The  following  two  statements  are  equivalent: 

(i)  For  every  pair  of  states  v,  u  £  V,  G  contains  a  stochastic  strategy  for  attaining  u  from  v. 

(ii)  A g  —  Sn~2,  withn=  \V\. 

Clarification:  In  (i),  the  strategy  may  depend  on  v  and  u ,  that  is,  different  pairs  of  states 
may  give  rise  to  different  strategies. 

Proof.  If  V  consists  of  a  single  state,  then  the  empty  simplex  is  a  stochastic  strategy 
for  attaining  that  state  from  itself.  In  fact,  that  is  the  only  possible  stochastically  acyclic 
collection  of  actions  of  G;  any  action  of  G  must  be  a  self-loop.  Consequently,  Ac  =  {0},  which 
by  convention  is  the  sphere  of  dimension  —1,  i.e. ,  n  —  2.  So,  henceforth  we  may  assume  that 
V  =  [n],  with  n  >  2. 

I.  Proof  that  (i)  implies  (ii): 

We  repeatedly  use  Lemma  9  to  infer  that  Uag.4  contains  every  point  of  Rn  except  the 
line  on  which  all  coordinates  are  equal.  Thus  A q  ~  Sn~2,  by  Theorem  8. 

II.  Proof  that  (ii)  implies  (i): 

Suppose  (i)  is  false.  Then  there  must  be  some  s  £  V  such  that  G  does  not  contain  a 
complete  stochastic  strategy  for  attaining  s  (on  the  state  space  V).  Define  a  new  stochastic 
graph  G+s  =  (V.  A'),  where  A1  is  the  union  of  A  and  all  possible  loopback  actions  at  s.  (This 
construction  is  similar  to  that  appearing  in  the  proof  of  Theorem  1,  except  that  we  have  added 
loopbacks  at  s  rather  than  merely  replaced  the  existing  actions  at  s  with  loopbacks.) 

Since  G+s  contains  all  the  actions  of  G, 

U  UA  C  U  UA- 

AeA  AeA' 

By  Theorem  8,  LUe.4  Ua  —  Sn~2. 

Using  the  fact  that  the  covering  sets  Ua  are  homogeneous  and  invariant  with  respect  to 
translation  along  the  line  {x  6  Rn  |  x\  =  •  •  •  =  xn},  as  well  as  the  fact  that  no  proper  subset 
of  a  sphere  is  homotopic  to  that  same  sphere,  one  sees  that  LUe-4'  ^ A  —  Sn~2. 
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On  the  other  hand,  a  collapsibility  argument  very  similar  to  that  used  in  the  proof  of 
Theorem  1  shows  that  Ag+s  must  be  contractible.  By  Theorem  8,  (J AeA'  Ua  has  the  same 
homotopy  type  as  Aq+s.  That  says  Sn~2  is  contractible,  a  contradiction.  □ 


Definition:  A  graph  is  fully  controllable  if,  for  any  initial  state  and  any  stop  state,  the  graph 
contains  a  stochastic  strategy  for  attaining  the  stop  state  from  the  initial  state. 

Example:  The  strategy  complex  of  the  graph  in  Fig.  13  is  honrotopic  to  a  circle,  i.e.,  to  Sn~2 . 
Every  action  in  the  graph  is  uncertain,  with  either  nondeterministic  or  stochastic  outcomes. 
Nonetheless,  Theorem  11  tells  us  that  the  graph  is  fully  controllable. 

Key  Point:  Despite  significant  control  uncertainty,  all  states  are  precisely  attainable. 

The  Power  of  Topology:  One  can  easily  verify  the  theorem’s  assertion  by  inspection  for 
the  example  of  Fig.  13.  For  instance,  the  system  can  be  certain  of  attaining  state  2  from  state 
1  via  the  strategy  {1  — >  p2,  q3  ;  3  — >  1,  2}.  What  we  do  not  know  is  the  exact  path  the  system 
will  take:  It  may  move  directly  to  state  2  from  state  1  or  it  may  move  through  state  3.  In 
fact,  it  may  even  cycle  for  a  while  between  states  1  and  3  before  moving  to  state  2.  This  is 
exactly  one  of  the  properties  we  sought  in  the  Introduction  (see  again  page  5):  When  planning 
in  the  presence  of  uncertainty,  one  should  focus  not  on  specific  trajectories  but  entire  classes 
of  motions.  Topology  is  doing  this  naturally  for  us. 

Here  is  a  time-bounded  version  of  Theorem  11: 

Theorem  12  (Time-Bounded  Graph  Controllability)  Let  G  =  (V,  A)  be  a  stochastic 
graph  with  V  ^  0  and  with  associated  nonnegative  action  transition  times  {5a} AeA-  Let,  T  >  0 
be  given.  The  following  two  statements  are  equivalent: 

(i)  For  every  pair  of  states  v,u  £  V,  G  contains  a  stochastic  strategy  crvu  for  attaining  u 
from  v  with  maximal  worst-case  expected  convergence  time  tmax(avu )  no  greater  than  T. 

(ii)  Aq  ~  Sn~2,  with  n  =  \V\. 

Proof.  The  proof  is  similar  to  that  given  for  Theorem  11.  One  difference  is  that  the  covering 
sets  U] 4  are  not  homogeneous  (they  are  still  invariant  with  respect  to  translation  along  the  line 
in  R"  on  which  all  coordinates  are  equal).  This  fact  is  one  reason  Lemma  7  uses  the  cylinder 
dCT+e-  □ 

Remark:  Theorem  1  in  Section  4  established  a  two-world  scenario  for  nondeterministic 
loopback  graphs:  A q<_3  is  honrotopic  either  to  Sn~2  or  to  a  point,  depending  on  whether 
G  contains  a  complete  guaranteed  strategy  for  attaining  s,  or  not,  respectively.  The  same 
result  holds  for  stochastic  loopback  graphs,  both  in  the  general  case  and  in  the  time-bounded 
case.  In  the  time-bounded  case,  we  associate  transition  time  zero  to  each  loopback  action. 
Thus,  A is  honrotopic  either  to  Sn~2  or  to  a  point.  If  G  contains  a  complete  stochastic 
strategy  for  attaining  s,  all  of  whose  worst-case  expected  convergence  times  are  bounded  by 
T,  then  A is  honrotopic  to  a  sphere.  Otherwise,  A 'q  is  honrotopic  to  a  point. 
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9  Topology  as  a  Design  Tool:  An  Example 


We  are  beginning  to  understand  how  the  topology  of  a  strategy  complex  reflects  a  system’s 
capabilities.  Theorem  11  tells  us  that  homotopy  equivalence  between  the  system’s  strategy 
complex  and  a  sphere  of  dimension  two  less  than  the  number  of  system  states  is  equivalent  to 
full  controllability. 

Moreover,  while  it  remains  a  research  question  to  understand  the  full  implications  of 
homotopy  equivalence,  we  will  soon  see  that  the  strategy  complex  precisely  characterizes 
the  stochastically  attainable  and  (potentially)  unattainable  goals  of  a  system.  Of  course, 
full  controllability  means  that  all  goals  are  stochastically  attainable.  In  the  absence  of  full 
controllability,  the  strategy  complex  still  informs  us  about  system  capabilities,  in  a  manner  to 
be  explained  in  the  next  section. 

This  section  illustrates  how  a  strategy  complex  may  be  used  as  a  design  tool:  We  turn 
design  knobs  while  watching  how  the  strategy  complex  changes,  freezing  the  design  when  the 
strategy  complex  exhibits  a  desired  topology. 


deterministic 


p=0 


stochastic 


o 

nondeterministic 


©.7— ►© 


o 

P=1 


© 

deterministic 


Figure  15:  The  parameter  space  for  designing  an  action  at  state  1.  The  tuning  parameter, 
drawn  in  green,  consists  of  an  interval  and  a  point.  The  action  is  depicted,  also  in  green,  for 
four  characteristic  parameter  values. 


Let  us  revisit  the  three-state  example  of  Fig.  13,  but  now  consider  the  situation  in  which  we 
are  designing  the  actions  and  the  control  error.  There  is  one  action  at  each  state.  The  action 
may  be  a  deterministic  transition  to  one  of  the  other  two  states,  or  a  stochastic  motion  to  both 
those  states,  with  tunable  transition  probabilities,  or  a  nondeterministic  motion  to  both  those 
states.  Fig.  15  shows  the  possible  tuning  parameter  for  the  action  at  state  1. 


9.1  How  Many  Design  Scenarios? 

Counting  the  design  scenarios  depends  on  one’s  perspective: 

•  Degrees  of  Freedom:  The  design  problem  is  a  three-degree-of-freedom  problem. 
Each  degree  of  freedom  may  be  modeled  as  [0, 1]  lj{<i},  where  [0, 1]  represents  the 
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stochastic  continuum  whose  endpoints  are  the  deterministic  actions  and  <  represents 
the  nondeterministic  action. 

•  Characteristic  Cases:  As  the  English  description  above  suggests,  there  are  four 
characteristic  cases  for  each  tuning  parameter,  so  64  characteristic  cases  overall.  Ignoring 
symmetries,  there  are  16  cases. 

Comment:  A  priori ,  the  precise  transition  probabilities  for  a  stochastic  action  could 
be  significant.  They  certainly  affect  convergence  times,  and  thus  the  complexes  A q. 
However,  varying  the  probabilities  does  not  affect  the  homotopy  type  of  Ac,  except 
possibly  at  an  endpoint  when  some  transition  probability  goes  to  zero. 

•  Topologically:  There  are  8  distinct  strategy  complexes  possible,  as  we  will  see  shortly. 
In  fact,  ignoring  symmetries,  there  are  only  4  topologically  distinct  cases.  In  other 
words,  from  the  perspective  of  overall  system  capabilities,  as  measured  by  stochastically 
attainable  goals,  this  design  problem  entails  a  choice  between  four  different  systems. 

9.2  Tuning  Convergence  Times  and  Designing  System  Capabilities 


x 

2 

i  L 


X 

1 

Figure  16:  One  may  design  system  capabilities  by  tuning  the  topology  of  A<£:  As  the  design 
parameters  change,  so  do  the  covering  sets  associated  with  the  graph’s  actions.  The  homotopy 
type  of  the  cover  intersected  with  d Ct+e  coincides  with  the  homotopy  type  of  the  simplicial 
complex  Aq.  This  figure  shows  a  snapshot  of  that  process.  As  did  Fig.  14,  the  figure  shows  a 
two-dimensional  slice  of  the  three-dimensional  covering  sets.  See  text  for  further  details. 
Color  Legend:  Corresponding  graph  actions,  covering  sets,  and  simplices  are  color-coded.  In  case 
color  is  not  visible,  placements  are  as  in  Figures  13  and  14. 

Probabilities:  The  actions  at  states  1  and  2  are  stochastic.  To  reduce  clutter,  the  figure  only  shows 
the  probabilities  of  moving  from  state  1  to  2  and  vice-versa. 


(2D  slice  at  x^=  1) 
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Fig.  16  shows  a  snapshot  of  the  tuning  process.  The  upper  right  frame  shows  the  graph 
and  its  actions  for  the  current  choice  of  tuning  parameters,  the  big  frame  on  the  left  shows  the 
(®i,  X2)-slice  at  X3  =  1  of  the  actions’  covering  sets.  As  in  Section  7,  each  action’s  covering 
set  gives  rise  to  a  vertex  of  the  simplicial  complex  A q,  shown  in  the  lower  right  frame  of  the 
figure.  Also  drawn  over  the  covering  sets  is  a  convergence  time  cylinder  dCx+t  for  some  desired 
convergence  time  T.  Key  intersection  points  of  the  covering  sets  within  this  desired  time  are 
highlighted.  They  give  rise  to  the  edges  in  the  complex  A j£;  the  thickness  of  an  edge  is  roughly 
proportional  to  the  difference  between  the  maximal  convergence  time  of  the  edge  (viewed  as  a 
stochastic  strategy)  and  the  maximal  time  T  permitted. 


Figure  17:  A  slice  of  the  design  space  from  the  perspective  of  the  resulting  stochastic  graphs.  Aj 
means  the  action  at  state  i.  In  the  slice  shown,  A3  is  nondeterministic  while  the  parameters  for 
actions  Ai  and  A2  vary  over  their  full  ranges.  For  the  stochastic  range  of  each  parameter,  the 
figure  depicts  a  representative  action.  To  reduce  clutter,  the  figure  only  shows  the  probability 
of  moving  from  state  1  to  2  or  vice-versa.  See  Fig.  18  for  the  associated  covering  sets  and 
Fig.  19  for  the  associated  simplicial  complexes. 


Figures  17-19  depict  a  two-dimensional  slice  of  the  three-degree-of-freedom  design  space 
from  three  perspectives.  In  this  slice,  the  tuning  parameter  for  the  action  at  state  3  is  fixed  to 
be  nondeterministic,  while  the  tuning  parameters  for  the  actions  at  states  1  and  2  vary.  Fig.  17 
shows  how  the  graphs  vary  as  the  tuning  parameters  vary,  Fig.  18  shows  how  the  covering  sets 
vary,  and  Fig.  19  shows  how  the  simplicial  complexes  Ac  vary  (we  now  ignore  the  precise 
convergence  times,  focusing  on  Aq  not  Aq).  Since  there  are  three  actions  without  self-loops, 
A q  always  contains  all  three  actions  as  vertices.  Aq  cannot  be  the  full  triangle  on  those  three 
vertices,  since  such  a  triangle  would  not  be  stochastically  acyclic.  Consequently,  there  should 
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1— >3  Aj  is  stochastic  1— >2  1— >2 ,3 


Figure  18:  A  slice  of  the  design  space  from  the  perspective  of  the  resulting  covering  sets. 
The  covering  sets  shown  correspond  to  the  graphs  of  Fig.  17,  assuming  all  actions  have  unit 
transition  time. 
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1— >3  Aj  is  stochastic  1— >2  1— >2,3 


Figure  19:  A  slice  of  the  design  space  from  the  perspective  of  the  resulting  simplicial  complexes 
A g-  The  complexes  shown  correspond  to  the  covering  sets  of  Fig.  18. 
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Figure  20:  There  are  four  topologically  distinct  simplicial  complexes  attainable  by  varying 
the  design  parameters  of  a  three-state  graph  with  one  (acyclic)  action  at  each  state.  These 
complexes  describe  the  inherent  capabilities  of  the  system  under  varying  degrees  of  control 
uncertainty.  The  columns  classify  the  64  characteristic  design  cases  in  terms  of  the  complexes 
they  generate.  The  leftmost  column  corresponds  to  full  controllability.  The  example  of  Fig.  13 
is  circled. 

Legend:  Each  characteristic  case  is  summarized  by  a  3-letter  code,  with  the  ith  letter  describing  the 
action  at  state  i  as  follows: 

S:  stochastic  m:  deterministic  transition  to  state  i  —  1 

N:  nondeterministic  p:  deterministic  transition  to  state  i  +  1 

(with  wraparound  at  3/1) 


be  as  many  different  complexes  in  the  full  design  space  as  there  are  ways  to  form  edges  from 
three  vertices,  namely  eight  ways.  Indeed,  inspection  of  Fig.  19  reveals  eight  unique  complexes; 
the  full  design  space  repeats  these.  Ignoring  symmetries,  there  are  in  fact  only  four  distinct 
complexes,  shown  in  the  top  row  of  Fig.  20. 

One  may  now  classify  each  of  the  64  characteristic  cases  of  the  design  space  by  the  type  of 
its  associated  strategy  complex  A<y,  as  shown  in  the  columns  of  Fig.  20. 

Section  10  discusses  in  more  detail  what  each  of  the  four  complexes  means.  We  already 
know  that  the  complex  defining  the  leftmost  column  constitutes  full  controllability.  We  thus 
see  twenty-seven  different  characteristic  implementations  that  attain  full  controllability.  The 
example  of  Fig.  13  is  circled. 
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Design  Implications:  From  a  design  perspective,  one  may  now  use  other  constraints  to 
decide  which  of  the  twenty-seven  implementations  might  be  desirable  for  full  controllability. 
Moreover,  one  may  tune  the  convergence  times  using  the  idea  described  in  Fig.  16.  This  process 
is  much  like  choosing  forces  within  the  nullspace  of  a  grasp  to  satisfy  some  design  criterion 
while  maintaining  a  specific  equilibrium  grasp  on  an  object  [51]. 

10  Duality 

In  order  to  understand  better  how  the  complex  AG  describes  system  capabilities,  we  need  to 
develop  a  perspective  dual  to  the  spherical  perspectives  of  Theorems  1  and  11.  In  the  process, 
we  will  see  how  backchaining  and  contractibility  of  certain  subcomplexes  are  manifestations  of 
the  same  idea.  Finally,  we  will  see  how  to  match  complexes  with  design  criteria. 

10.1  Start  Region  Contractibility 

Consider  the  start  region  Va  of  some  simplex  a  of  stochastically  acyclic  actions  in  a  graph 
G.  As  we  will  see  in  this  section,  the  subcomplex  induced  by  all  actions  of  the  graph  with 
sources  in  that  region  is  contractible.  Intuitively,  contractibility  is  consistent  with  viewing  a 
as  a  collapsing  of  Va,  moving  the  system  off  those  states. 

Definitions  Suppose  G  =  (V,A)  is  a  stochastic  graph  and  IF  C  F.  Define  the  following: 

•  A\W  is  the  collection  of  all  actions  of  G  whose  sources  lie  in  IF. 

•  Wa  is  the  union  of  IF  and  all  targets  of  actions  in  A]  IF. 

•  G\W  =  (IF4,  A\ IF) .  Intuition:  G\W  is  a  subgraph  of  G,  induced  by  all  the  actions 
of  G  whose  sources  lie  in  IF  but  whose  targets  may  lie  outside  of  IF. 

•  A  moves  off  IF  if  A  6  A\W  and  one  of  the  following  is  true: 

(i)  A  is  stochastic  with  at  least  one  of  its  targets  in  V  \  IF,  or 

(ii)  A  is  nondeterministic  with  all  of  its  targets  in  V  \  IF. 

Lemma  13  (Contractibility  of  Start  Regions)  Let  G  =  (F,  A)  be  a  stochastic  graph. 
Suppose  W  =  Va  for  some  a  G  A^.  Then  AG\W  is  contractible. 

Proof.  If  IF  =  0,  then  AG\W  is  the  void  complex  0,  which  is  considered  contractible  [47] 
(do  not  confuse  it  with  the  empty  complex  {0}). 

If  IF  A  0,  then  there  there  is  some  action  A  £  A\W  that  moves  off  IF.  To  see  this,  suppose 
otherwise.  Imagine  constructing  a  Markov  chain  M  with  support  IF  from  the  graph  (F,  a)  via 
the  process  of  page  22.  If  no  action  A  moves  off  IF,  then  an  adversary  can  ensure  that  M  has 
no  transitions  to  states  outside  of  IF.  This  means  M  must  contain  a  recurrent  class  within  IF, 
since  the  chain  is  finite  [32],  That  contradicts  the  stochastic  acyclicity  of  a  (recall  Lemma  5). 

Now  suppose  r  G  AG\W.  Consider  t'  =  tU{A}.  If  t'  is  not  stochastically  acyclic,  then 
t'  must  give  rise  to  some  Markov  chain  (M,  IF ')  whose  support  IF'  is  a  recurrent  class.  Since 
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r  is  stochastically  acyclic,  M  must  include  transitions  induced  by  A.  Since  A  moves  off  W  it 
moves  off  W' ,  contradicting  the  assumed  recurrent  nature  of  W' . 

Consequently,  t'  is  stochastically  acyclic,  establishing  that  A q\w  is  a  cone  with  apex  A, 
hence  contractible.  □ 

Remark:  A  similar  result  holds  for  the  time-bounded  case.  The  proof  is  more  involved.  The 
problem  in  the  time-bounded  case  is  that  the  action  A  appearing  in  the  proof  above  need  no 
longer  be  a  cone  apex  for  the  complex  A^^.  The  reason  is  that  some  worst-case  expected 
convergence  time  of  r'  may  exceed  the  desired  bound  T,  even  though  r  and  A  separately  satisfy 
the  bound.  One  must  therefore  argue  differently.  In  particular,  one  can  return  to  the  covering 
set  approach  and  see  that  the  cover  corresponding  to  A^^  must  be  contractible. 

10.2  Source  Complex 

This  section  associates  a  new  simplicial  complex  to  every  graph,  called  the  source  complex ,  in 
effect  compressing  the  graph’s  strategy  complex.  The  main  result  is  that  compression  preserves 
homotopy  type.  This  result  explains  a  slight  sleight  of  hand  in  the  discussion  of  Section  9.  Even 
though  strategy  complexes  reside  in  a  space  of  actions,  we  managed  to  view  the  complexes  as 
sitting  on  the  original  graph.  Doing  so  was  very  natural  in  the  example  of  Section  9  since  the 
graph  contained  exactly  one  action  at  each  state.  The  current  section  establishes  that  one  may 
always  view  the  strategy  complex  as  residing  back  on  the  graph,  via  the  compression  to  source 
complexes. 

Definition:  The  source  complex  Aq  of  a  stochastic  graph  G  =  ( V. ,  A)  is  the  collection 
{Va  |  a  G  Ac}.  The  underlying  vertex  set  of  Aq  is  V. 

It  is  easy  to  check  that  A q  really  is  a  simplicial  complex. 

Aq  describes  all  possible  start  regions  of  strategies  definable  by  actions  of  G.  The 
complements  of  those  start  regions,  that  is,  sets  of  the  form  V  \  Va.  describe  all  stochastically 
attainable  goals.  Observe  that  V  is  never  a  simplex  of  Aq,  since  any  purported  strategy  with 
actions  at  all  states  of  a  graph  would  actually  cycle  forever. 

Theorem  14  (Compression  Preserves  Homotopy  Type)  Let  G  be  a  stochastic  graph. 

Ag  ~  Aq. 

Proof.  Let  P  =  P(Aq)  and  Q  =  P(A q)  be  the  face  posets  of  the  two  complexes. 

Define  /  :  P  — >  Q  by  f(a)  =  Va.  Suppose  W  G  Q.  Then  f~1(Q<w )  is  the  face  poset  of 
Aq\w,  which  is  contractible  by  Lemma  13.  The  desired  result  now  follows  from  the  Quillen 
Fiber  Lemma.  □ 

Remark:  Again,  a  similar  result  holds  for  the  time-bounded  case. 

The  following  lemma  is  a  useful  tool: 

Lemma  15  (Source  Complex  Membership)  Let  G  =  (V,  A)  be  a  stochastic  graph. 
Suppose  W  is  a  nonempty  proper  subset  of  V  such  that  every  proper  subset  ofW  is  a  simplex 
in  A q.  Then  W  G  A^  if  and  only  if  G  contains  an  action  that  moves  off  W. 
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Proof.  One  direction  follows  from  the  proof  of  Lemma  13. 

For  the  other  direction,  suppose  A  moves  off  IF.  Let  v  be  the  source  of  A.  By  assumption, 
there  is  some  a  G  AG  such  that  Va  =  IF  \  {u}.  Consider  r  =  cr|J{A}.  r  is  stochastically 
acyclic  since  a  is  stochastically  acyclic  and  since  any  transitions  at  v  induced  by  A  in  a  Markov 
chain  derived  from  r  have  probability  less  than  1  of  remaining  in  IF.  VT  =  IF,  so  IF  G  A G. 
□ 

10.3  Contractibility  Characterization  of  Goal  Attainability 

We  now  obtain  a  result  dual  to  Theorem  1.  The  proof  of  the  following  theorem  makes  explicit 
the  connection  between  backchaining  and  contractibility. 

Definition:  Suppose  S  is  a  simplicial  complex  with  underlying  vertex  set  V.  A  minimal 
nonface  of  S  is  a  set  IF  C  V  such  that  IF  is  not  a  simplex  of  S  but  every  proper  subset  of  IF 
is  a  simplex  of  S. 

Theorem  16  (Contractibility  Characterization  of  Goal  Attainability) 

Suppose  G  =  (V,  A)  is  a  stochastic  graph  and  0/5  C  F.  S  is  a  stochastically  attainable 
goal  in  G  if  and  only  if  AG\W  is  contractible  for  every  IF  C  V  \  S. 

Proof.  I.  Suppose  G  contains  a  complete  stochastic  strategy  for  attaining  SCf. 

Then  there  is  some  a  G  such  that  Va  =  V  \  S.  By  Lemma  13,  and  the  simplicial  nature 
of  a ,  we  see  that  AG\W  is  contractible  for  every  IF  C  TA¬ 
IL  Suppose  Aq\w  is  contractible  for  every  IF  C  V  \  S. 

We  now  construct  a  G  A q  by  backchaining: 

1.  Initialize  S*  :=  S  and  o  :=  0. 

2.  If  S*  =  V,  then  done;  return  a. 

3.  Otherwise,  as  shown  below,  G  contains  an  action  A  that  moves  off  V  \  S*. 

Let  v  be  the  source  of  A.  Update  S*  :=  S*  U  {u}  and  a  :=  cr  [j  {A}. 

4.  Repeat  Steps  2  and  3  until  done. 

It  is  easy  to  see  that  the  a  returned  in  Step  2  really  is  stochastically  acyclic.  Va  =  V  \  S, 
so  a  is  a  complete  stochastic  strategy  for  attaining  S,  as  desired. 

To  complete  the  proof,  we  need  to  establish  the  existence  of  action  A  in  Step  3.  Suppose 
no  such  A  exists.  By  the  proof  of  Lemma  13,  this  means  V  \  S*  fL  A G.  Choose  nonempty 
W  C  V  \S*  such  that  W  A q  but  every  proper  subset  of  IF  is  a  simplex  of  A q.  In  other 
words,  IF  is  a  minimal  nonface  of  A q.  Observe  that  IF  is  also  a  minimal  nonface  of  A^i^. 

A  priori ,  the  underlying  vertex  set  of  AG\W  is  the  state  space  W4  of  G\W ,  but  no  state 
outside  IF  can  be  a  vertex  of  AGiw  since  G\W  contains  no  actions  at  such  states.  Consequently, 
IF  being  a  minimal  nonface,  AG\\y  is  the  boundary  complex  of  the  full  simplex  on  IF.  Thus 
AG\w  ~  l~2.  On  the  other  hand,  Theorem  14  implies  AG\W  ~  AG\W,  which  is  contractible. 

Contradiction.  □ 
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10.4  The  Dual  Complex 

In  algebraic  topology,  geometric  duality  becomes  algebraic  duality.  The  theorems  are  often 
formulated  for  nonempty  proper  subsets  of  spheres.  Given  the  importance  of  spheres  when 
reasoning  about  strategy  complexes,  one  imagines  that  duality  might  help  illuminate  the 
homotopy  type  of  a  strategy  complex.  There  is  a  simple  combinatorial  description  for  simplicial 
complexes  of  a  particular  duality  known  as  Alexander  Duality  [7].  Formally,  given  a  simplicial 
complex  S  with  underlying  vertex  set  V,  its  Combinatorial  Alexander  Dual  is  the  complex 
£*  =  {a  C  V  |  V  \  a  0  £}.  Observe  that  (£*)*  =  S. 

The  Alexander  dual  of  a  source  complex  has  an  important  natural  meaning:  it  describes 
all  goals  that  are  not  stochastically  attainable  (in  the  sense  of  page  30),  as  follows. 

_ * 

Definition:  The  dual  complex  AG  of  a  stochastic  graph  G  =  ( V. ,  A)  is  the  collection 
jV*  C  V  V  \  V*  0  Aq  j.  The  underlying  vertex  set  of  AG  is  again  V. 

_ * 

It  is  again  easy  to  verify  that  AG  is  a  simplicial  complex.  Moreover,  it  follows  from  the 
_ * 

definitions  that  V*  £  AG  if  and  only  if  there  is  no  strategy  a  £  AG  such  that  V  \  Va  C  V*.  In 

_ * 

other  words,  we  may  think  of  AG  as  encoding  all  the  potentially  unattainable  goals,  meaning 
there  is  no  stochastic  strategy  for  attaining  such  a  goal  from  everywhere  in  the  graph. 

The  reader  may  be  wondering  why  we  have  not  also  defined  a  simplicial  complex  to  define 
the  stochastically  attainable  goals.  The  answer  is  that  simplicial  complexes  must  satisfy  some 
kind  of  monotone  property:  subsets  of  simplices  must  also  be  simplices.  Failure  to  attain  a 
goal  is  a  monotone  property;  if  we  cannot  be  certain  of  attaining  a  goal  of  some  size  when 
planning  with  uncertainty,  then  we  also  cannot  be  certain  of  attaining  any  subset  of  that  goal. 
In  contrast,  attainability  is  not  similarly  monotone;  just  because  we  can  attain  some  goal  does 
not  mean  we  can  attain  a  more  precisely  defined  goal. 

In  short,  start  regions  and  potentially  unattainable  goals  define  simplicial  complexes  via 
the  source  and  dual  complexes.  The  stochastically  attainable  goals  are  given  implicitly,  as  the 
complements  of  the  start  regions. 

10.5  Duality  in  Design 

We  now  understand  better  the  topological  information  encoded  in  a  graph’s  strategy  complex. 
Earlier  (Fig.  11)  we  saw  that  homotopy  equivalence  naturally  favors  precise  actions  over 
imprecise  actions.  We  also  realized  (page  35)  that  topology  abstracts  away  from  particular 
trajectories  to  broad  classes  of  motions.  The  source  and  dual  complexes  make  this  explicit. 
Moreover,  the  state  space  V  of  a  graph  provides  a  good  reference  frame  onto  which  homotopy 
equivalences  should  map.  One  can  compare  different  system  capabilities  of  different  graphs 
with  underlying  state  space  V  by  comparing  their  source  (and/or  dual)  complexes. 

Returning  to  the  design  problem  of  Section  9,  Fig.  21  now  shows  both  the  source  and  dual 
complexes  corresponding  to  Fig.  20.  Moreover,  the  figure  makes  sense  even  if  there  are  multiple 
actions  or  design  parameters  at  each  state. 

Recall  the  various  semantics  we  have  been  attaching  to  complexes.  Consider  the  first 
column  of  complexes  in  Fig.  21.  Previously  we  observed  that  this  column  corresponds  to  full 
controllability  since  the  strategy  complex  is  homotopic  to  a  circle.  That  statement  still  holds, 
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Figure  21:  A  comparison  of  the  source  and  dual  complexes  for  the  design  problem  of  Fig.  20. 


only  now  we  may  refer  to  the  source  complex,  since  we  now  know  that  the  strategy  and  source 
complexes  have  the  same  homotopy  type.  The  dual  complex  in  this  case  is  the  empty  complex, 
meaning  that  there  are  no  potentially  unattainable  goals.  That  statement  is  consistent  with 
full  controllability:  all  goals  are  stochastically  attainable. 

Remark:  The  reader  may  observe  that  this  reasoning  provides  the  basis  for  a  purely 
combinatorial  proof  of  Theorem  11,  made  possible  by  Theorem  14  and  the  semantics  of  source 
and  dual  complexes. 

In  some  cases,  one  may  not  want  full  controllability.  For  instance,  imagine  that  we  are 
designing  the  hallways  and  door  controllers  in  a  bank.  Suppose,  for  simplicity,  there  are  three 
states:  Outside,  Lobby,  and  Vault.  We  might  want  to  ensure  that  Outside  and  Lobby 
are  reachable  from  everywhere  by  everyone,  but  that  Vault  is  reachable  only  by  designated 
people.  This  suggests  a  system  whose  capabilities  switch  between  those  given  by  the  first  and 
third  columns  of  complexes  in  Fig.  21,  depending  on  the  people  walking  in  the  hallways.  We 
could  go  back  to  Fig.  20  to  select  a  particular  implementation  of  these  switchable  capabilities 
that  also  avoids  passing  by  the  Vault  accidentally.  For  instance,  we  might  use  a  switchable 
deterministic  action  in  the  Lobby  whose  actual  direction  is  under  the  control  of  the  bank’s 
security  guards.  Finally,  we  might  also  go  back  to  Fig.  16  to  fine-tune  convergence  times. 
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11  Modularity 


This  section  discusses  combination  and  simplification  of  graphs. 

First,  some  topology.  The  (topological)  join  X  *  Y  of  two  topological  spaces  X  and  Y  is 
the  quotient  space  obtained  from  X  x  Y  x  [0, 1]  by  identifying  each  set  {x}  x  Y  x  {0}  to  a 
point  and  each  set  X  x  {y}  x  {1}  to  a  point,  for  all  x  G  X  and  y  E  Y  [61].  Geometrically,  one 
may  think  of  X  *  Y  as  the  union  of  all  line  segments  joining  points  in  X  to  points  in  Y  [39]. 
For  instance,  the  topological  join  of  two  finite  disjoint  edges  is  a  tetrahedron. 

In  the  case  of  simplicial  complexes,  the  join  assumes  a  simple  combinatorial  form: 

Definition:  Suppose  S  and  F  are  two  simplicial  complexes  with  disjoint  underlying  vertex 
sets.  Their  (simplicial)  join  is  the  simplicial  complex  defined  as 


s  *r 


a  E  S  and  7  E 


For  instance,  if  S  and  F  represent  two  disjoint  line  segments,  say 


£  =  {0,{PlMP2},{Pl;P2}}, 

r  =  {0,  {<?i},  {^2},  {<?i,  ^2}}, 


then 


s*r  =  {0,  {pi},  {P2},  {<7i},  {<?2}, 

{Pi,P2},  Ui,q2},  {Pi,  <?i},  {pi,  Q2},  {P2,Qi},  {p2 1  q2}, 

{Pi,P2,qi},  {pi,P2,  12},  {pi,qi,q2},  { P2 ,  qi,q2},  {pi,P2,  <?i,  <12}}, 

which  represents  a  tetrahedron. 

Observe:  S  *  {0}  =  {0}  *  £  =  S  and  E  *  0  =  0  *  E  =  0,  for  all  simplicial  complexes  E. 
Fact:  Homotopy  equivalence  (~)  commutes  with  the  join  operator  (*). 

11.1  Graph  Union 

Suppose  G 1  =  (U1M1)  and  G2  =  (V2M2)  are  two  stochastic  graphs.  (The  state  spaces  Vj  and 
V2  are  allowed  to  overlap.) 

Consider  the  stochastic  graph  G  =  (V,  „4)  representing  the  union  of  G 1  and  G 2.  It  is  defined 
by  V  =  Vj  U  ^2  and  A  =  Ai  U  A2-  The  operator  “U”  means  “disjoint  union”,  that  is,  we  treat 
actions  in  Ai  and  A2  as  distinct  even  if  they  happen  to  have  identical  edge  sets.  (Any  resulting 
redundancy  washes  away  under  homotopy  equivalence,  as  we  saw  in  Section  5.5  and  Fig.  11.) 
We  write  G  =  G±  (J  G2  for  shorthand. 

We  wish  to  understand  the  relationship  between  Ac,  A^q,  and  A g2.  Let  us  assume  that  V  j 
and  Vj  are  both  nonempty.  Treating  A\  and  A2  as  disjoint  means  A^q  and  Aq2  have  disjoint 
underlying  vertex  sets,  so  we  can  form  their  join.  Observe  that  A^  C  A^q  *  A g2,  in  fact, 

Ag  =  [t£  Aq1  *  Ag2  I  t  is  stochastically  acyclic}. 
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The  next  lemma  describes  some  common  cases  in  which  there  is  equality  of  complexes. 
Case  (a)  covers  completely  disjoint  graphs,  case  (b)  covers  graphs  that  touch  at  a  single  state, 
and  case  (c)  covers  graphs  in  which  one  graph  is  essentially  feeding  into  the  other. 

Lemma  17  (Join  Sufficiency)  Suppose  G  =  G \  (J  G2 ,  with  notation  as  above.  Assume  that 
V\  and  V2  are  both  nonempty.  In  each  of  the  following  cases,  A q  =  Aq1  *  Aq2  : 

(a)  V1f)V2  =  0, 

(b)  Ij  fj  V2  contains  a  single  state, 

(c)  G 1  (or  G2)  has  no  actions  with  sources  in  Ij  f]V2. 

Proof,  (a)  Clear  from  the  definitions.  (Also,  (a)  follows  from  (c).) 

(b)  If  a  (J  7  is  not  stochastically  acyclic  although  a  £  Agq  and  7  £  Aq2  ,  then  a  IJ  7  must 
induce  a  Markov  chain  (M,  IT)  whose  support  IT  is  a  recurrent  class  containing  the  common 
state  of  V\  and  V2.  The  transitions  of  M  at  that  state  are  induced  by  a  single  action  A,  with 
either  A  £  a  or  A  £  7.  In  the  first  case,  all  of  A’s  targets  lie  in  Vi  and  so  a  alone  could 
generate  (M,  IT),  contradicting  the  stochastic  acyclicity  of  a.  Similarly  for  the  second  case. 

(c)  Suppose  the  sources  of  all  of  G 1  ’s  actions  lie  in  V\  \  V2.  If  <7  (J  7  is  not  stochastically 
acyclic  although  a  £  Ag1  and  7  £  Aq2,  then  a  |J  7  must  induce  a  Markov  chain  (M,  IT)  whose 
support  IT  is  a  recurrent  class  overlapping  both  V\  \  V2  and  V2.  Any  action  with  source  in  V2 
must  be  an  action  of  7  and  thus  has  no  targets  in  Ij  \  V2,  contradicting  the  assumption  that 
IT  is  a  recurrent  class  of  M.  □ 

11.2  Testing  Acyclicity 

Suppose  G±  and  G2  do  not  satisfy  the  conditions  of  Lemma  17.  The  following  options  for 
computing  Aq  exist: 

1.  One  possibility  is  to  compute  A^  rather  than  A^.  Conceptually,  A^  is  easier  to  compute 
than  Aq-  The  two  complexes  have  the  same  homotopy  type  (Theorem  14).  Moreover, 
A g  is  more  explicitly  useful  in  characterizing  system  capabilities,  as  we  saw  in  Section 
10.  Computing  Aq  amounts  to  repeated  backchaining  (see  Section  12). 

2.  Another  possibility  is  to  work  with  the  definitions  directly.  For  instance,  in  deciding 
whether  a  potential  simplex  cr|J7  really  is  stochastically  acyclic,  one  may  attempt  to 
solve  System  (1)  (written  out  for  ctUt)  using  strictly  positive  action  transition  times 
{cM}-  The  solution  will  diverge  precisely  when  a  |J  7  can  induce  a  Markov  chain  whose 
support  is  a  recurrent  class. 

3.  Finally,  a  third  possibility  is  to  break  the  computations  into  simpler  pieces  using  the 
tools  described  next. 

Lemma  18  (Combining  Actions)  Let  G  =  (V,A)  be  a  stochastic  graph  and  a  £  Aq- 
Suppose  there  are  actions  A\, . . .  ,A^  £  A,  all  with  the  same  source,  such  that  a  (J  {A}  £  A g 
for  each  individual  action  Ai,  i  =  1, . . . ,  k.  Then  cr  (J  {A\, . . . ,  A}  £  A g- 
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Proof.  Let  v  be  the  source  of  the  actions  Ai, . . . ,  A*.. 

Any  Markov  chain  ( M,W )  induced  by  a  (J  {A\, . ..,  A*,}  whose  support  W  is  a  recurrent 
class  must  have  transitions  at  v  induced  by  some  A*.  But  then  cr  (J  {A;}  could  induce  the  same 
Markov  chain.  Contradiction.  □ 


Now  suppose  a  £  Aq  and  A  is  an  action  of  G.  To  avoid  trivialities,  assume  A  is  not 
a  self-looping  (non) deterministic  action.  Let  va  be  A’s  source.  One  can  decide  whether 
cr  (J  {A}  £  A q  as  follows: 

•  By  Lemma  18,  one  may  assume  without  loss  of  generality  that  va  0  14  (recall  14  is  the 
set  of  all  sources  of  actions  in  cr). 

•  By  Lemma  17(c),  if  va  is  not  the  target  of  some  action  of  cr,  or  if  no  target  of  A  lies  in 
14,  then  cr(J  {A}  £  Ac-  (This  step  is  not  necessary  but  provides  a  convenient  filter.) 

•  Now  consider  the  following  system  of  equations: 


qv  =  max 


max 
B  £  a 
B  =  {(v,«j)} 


|^max  qUj 


V 

qv  =  i,  if  v  =  vA ; 

qv  =  0,  otherwise. 


max 

B  £  cr  \  3 
B  =  {( v,pjUj )} 


Y 


,  if  v  £  14; 


(5) 


This  system  has  a  unique  finite  solution  {qv}v^v,  since  a  is  stochastically  acyclic.  One 
can  compute  the  solution  using  iteration.  For  each  v  £  V,  qv  is  the  maximum  probability 
under  all  Markov  chains  induced  by  cr  that  the  system,  when  started  at  state  v,  will  reach 
va,  the  source  of  action  A. 


Consequently: 


(i)  If  A  is  stochastic  then  a  IJ  {A}  £  Aq  if  and  only  if  qu  <  1  for  some  target  u  of  A. 

(ii)  If  A  is  nondeterministic,  then  a  (J  {A}  £  Aq  if  and  only  if  qu  <  1  for  all  targets  u 
of  A. 


11.3  Simplification  via  Strongly  Controllable  Subspaces 

For  regular  directed  graphs,  one  defines  an  equivalence  relation  by  saying  that  two  states  are 
equivalent  if  each  is  reachable  from  the  other.  The  resulting  equivalence  classes  are  called 
strongly  connected  components.  One  may  form  a  quotient  graph  by  collapsing  each  strongly 
connected  component  to  a  single  point.  Ignoring  induced  self-loops,  the  result  is  a  directed 
acyclic  graph. 

This  section  pursues  such  simplification  for  stochastic  graphs.  We  will  discover  some 
enrichments  not  present  in  regular  directed  graphs. 
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Figure  22:  States  1  and  3  are  each  certainly  attainable  from  the  other,  but  an  excursion  to 
state  2  cannot  be  either  excluded  or  forced  by  the  system  when  leaving  state  1.  (An  adversary 
may  have  control  over  the  precise  path.) 

An  Equivalence  Relation 

The  first  subtlety  concerns  connectivity.  It  is  not  enough  to  define  two  states  as  equivalent  if 
each  is  reachable  from  the  other.  For  instance,  in  Fig.  22,  there  exist  strategies  for  attaining 
state  3  from  state  1  and  vice-versa.  The  problem  is  that  the  strategy  to  attain  3  from  1  might 
or  might  not  pass  through  state  2.  That  would  be  fine  if  the  graph  contained  strategies  for 
purposefully  attaining  state  2  from  1  and  3,  but  it  does  not. 

We  therefore  need  a  stronger  requirement  than  pairwise  reachability.  Here  it  is: 

Definitions  Suppose  G  =  (V,A)  is  a  stochastic  graph. 

Define  a  binary  relation  on  V  x  V  as  follows: 

•  v  w  if  G  contains  a  subgraph  H  =  (IF,  B)  such  that: 

(a)  v,  w  £  W,  and 

(b)  A H  ~ 

It  is  easy  to  see  that  defines  an  equivalence  relation  on  V .  We  refer  to  an  equivalence 
class  of  as  a  strongly  controllable  subspace  of  G. 

For  each  equivalence  class  IF  there  is  some  subgraph  H  =  (IF,  B )  whose  strategy  complex 
is  homotopy  equivalent  to  a  sphere  of  dimension  |IF|— 2.  The  proof  of  Theorem  11  ensures 
that  H  contains  strategies  for  attaining  any  state  in  IF,  without  leaving  IF. 

•  If  ~  is  any  equivalence  relation  on  V,  let  G/~  be  the  quotient  graph  obtained 

by  collapsing  states  to  their  equivalence  classes.  Thus  collapses  the  strongly 

controllable  subspaces  of  G. 

A  special  case:  Let  H  =  (IF,  B)  be  a  subgraph  of  G  with  IF  /  0.  Define  the  equivalence  ~ 
as  follows:  Each  state  of  V \  IF  is  equivalent  only  to  itself,  while  all  states  in  IF  are  equivalent. 
This  means  IF  collapses  to  a  point,  call  it  o.  We  designate  the  resulting  quotient  graph  by 
G/W.  G/W  =  (V',  A'),  with  V'  =  V\  W  (J  {o}  and  A !  essentially  identical  to  A  except  that 
any  states  of  IF  appearing  as  sources  or  targets  in  A  have  been  remapped  to  o  in  A1 . 

Side  issues:  Distinct  actions  of  G  may  effectively  become  the  same  action  in  G/~. 
Nonetheless,  we  treat  them  as  distinct.  In  other  words,  there  is  a  bijective  correspondence 
between  A  and  A! .  Once  again,  any  redundancy  will  be  washed  away  by  homotopy  equivalence 
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once  we  pass  to  strategy  complexes.  The  reader  may  also  notice  that  for  a  given  action,  different 
targets  in  G  may  become  the  same  target  in  G/~.  This  poses  no  real  issue.  Nondeterministic 
actions  are  sets  of  edges,  so  the  redundancy  disappears  automatically.  For  stochastic  actions, 
one  can  add  up  the  probabilities  of  different  edges  with  the  same  target  to  form  a  single 
edge.  Finally,  some  actions  that  are  stochastically  acyclic  in  G  may  very  well  become  self¬ 
looping  (non) deterministic  actions  in  G'/~.  Such  actions  disappear  when  we  pass  to  strategy 
complexes. 

Now  imagine  that  H  =  ( W,B )  is  a  subgraph  of  G  satisfying  A h  —  S^w^~2  (W  need  not  be 
an  equivalence  class  of  G  under  <->,  but  must  certainly  be  a  subset  of  such  an  equivalence  class). 
Then  the  system  has  full  controllability  within  H,  by  Theorem  11.  Consequently,  H  and  G/W 
are  almost  separate  graphs  that  touch  at  a  single  state,  as  in  Lemma  17(b).  We  might  therefore 
expect  to  see  equality  between  complexes  as  in  that  lemma.  This  analogy  is  not  quite  correct; 
the  single  common  state  is  o  in  G/W,  but  it  might  be  all  of  W  in  H.  As  a  result,  equality 
becomes  homotopy  equivalence,  as  the  following  lemma  and  its  proof  demonstrate. 

Lemma  19  (Factoring  Controllable  Subgraphs)  Let  G  =  ( V. ,  A)  be  a  stochastic  graph 
and  H  =  (W,  B)  a  subgraph  satisfying  A h  —  S^WY2,  with  W  /  0.  Then 


Ac  —  A H  *  Aq/w. 


Proof.  By  Theorem  14,  we  need  merely  prove  Aq  —  A u  *  A g/w- 

Let  P  =  F(Ah  *  Ag/w)  and  Q  =  ^(Ag)  be  the  associated  face  posets.  Every  p  in  P  is  a 
nonempty  simplex  of  Ah  *  A q/w,  so  we  will  simply  write  X  (J  Y  for  elements  of  P. 


Define  /  :  P  — »  Q  by 


X  U  Y,  ifo0T; 

W{JY\{o},  if  067; 


with  X  G  A h  and  Y  e  AG/W.  Observe  that  X  C  W  and  Y  C  V  \  W  (J  {o}. 

Recall:  o  is  the  state  of  G/W  that  represents  W  collapsed  to  a  point. 

[Comment:  Observe  the  power  of  posets:  /  maps  a  singleton  set  {o}  to  all  of  W .} 

The  Quillen  Fiber  Lemma  will  give  us  the  desired  result,  if  we  can  satisfy  these 
preconditions: 

(i)  /  is  well-defined,  meaning  that  f(X  (J  Y)  really  is  a  simplex  of  A  o', 

(ii)  /  is  order-preserving; 

(iii)  the  fibers  f~1{Q<q)  are  contractible. 


Establishing  these  three  conditions  is  a  bit  tedious.  There  are  several  cases.  We  will  prove 
the  most  interesting  cases  and  leave  the  rest  to  the  reader.  We  emphasize  that  one  really  must 
prove  (i);  full  controllability  within  H  is  needed  to  establish  that  /  is  well-defined. 
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(i) .  We  assume  o  £  Y .  We  leave  to  the  reader  the  case  in  which  o  0  Y . 

Let  X  £  Ah  and  Y  £  AG/w  be  given,  with  o  6  Y .  This  means  there  exists  a  stochastically 
acyclic  set  of  actions  t'  £  A G/w  such  that  VT>  =  Y.  Now  let  r  be  the  actions  of  G  that  generate 
t'  (recall:  the  correspondence  between  actions  in  G  and  G/W  is  bijective).  Since  o  £  Y,  t' 
includes  an  action  A'  with  source  o.  We  may  assume  there  is  exactly  one  such  action  in  t'  . 
Let  wq  £  W  be  the  source  of  the  corresponding  action  A  of  r.  Since  A h  —  S^w  l~2,  the  proof 
of  Theorem  11  implies  a  stochastically  acyclic  set  of  actions  a  £  A h  such  that  Va  =  W\{r(;o}- 
It  is  not  hard  to  see  that  cj\Jt  is  stochastically  acyclic  since  t'  is  (Lemma  17(c)  makes  part  of 
the  argument,  establishing  that  \  {T}  is  stochastically  acyclic).  Finally, 

Va[jT  =  W\{w0}  |J  Y\{o}  |J  {w0}  =  W\jY\{o}, 

meaning  f(X\JY)  £  AG. 

(ii) .  Easy. 

(iii) .  Every  q  £  Q  is  a  nonempty  simplex  of  A^,  so  we  will  write  V  in  place  of  q,  with 

V  C  V.  Let  V  be  given.  We  need  to  show  that  f~1(Q<y)  is  contractible. 

We  deal  here  with  the  case  in  which  V  f)W  =  W.  We  leave  to  the  reader  the  case  in  which 

V  H  W  is  a  proper  subset  of  W. 

To  establish  contractibility,  we  will  show  that  f~1{Q<y)  is  the  face  poset  of  a  cone  with 
apex  o.  Observe  that  f~1{Q<y)  is  indeed  the  face  poset  of  a  simplicial  complex,  since  /  is 
order-preserving. 

Let  XU  Y  £  f~1(Q<y),  with  X  £  A h  and  Y  £  A q/w-  Suppose  o  0  Y.  We  need  to  show 
that  XU  Y  U  {o}  is  an  element  of  f~1(Q<y)-  Observe: 

/(XU  30  =  X{JY  c  V,  since  XU  X£/"1(Q<y); 

/(XUL'UH)  =  W[jY  C  v,  since  W  CV. 

There  remains  to  show  that  LUW  £  A g/w-  Suppose  that  is  false.  Then  there  must  be 
some  set  of  states  Y'  C  Y,  such  that  every  proper  subset  of  Y'  U  {^}  is  a  simplex  of  AG/W  but 
Y'  U  {^}  is  n°L  By  Lemma  15,  no  action  of  G/W  moves  off  Y'  U  {^},  which  means  no  action 
of  G  moves  off  Y’\JW.  On  the  other  hand,  Y'\JW  AY\JW  C  V,  implying  Y'  U  W  £  Aq- 
Now  Lemma  15  reveals  a  contradiction. 

The  same  argument  shows  that  {o}  is  itself  in  f~1{Q<y)-  □ 

Theorem  20  (Controllability  Structure)  Let  G  be  a  stochastic  graph.  Then 

AG  ~  gn-k-l  ,  Ag/mj 

where  n  is  the  size  of  G’s  state  space  and  k  is  the  number  of  equivalence  classes  induced  by  <->. 

Proof.  Use  Lemma  19  repeatedly  on  the  equivalence  classes  determined  by  <->.  Bear  in 
mind  that  Si  *  Sj  ~  Si+j+1  [6].  □ 

Theorem  20  generalizes  a  result  of  Hultman’s  for  directed  graphs  [44],  For  directed  graphs, 
it  is  not  hard  to  see  that  AG /<_>  is  either  the  empty  complex  or  homotopic  to  a  point.  So 
for  directed  graphs,  AG  always  looks  like  a  sphere  or  a  point,  homotopically.  For  stochastic 
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(in  particular,  nondeterministic)  graphs,  the  domain  is  considerably  richer  (see  Section  13  for 
instance);  A q/^  may  have  arbitrary  homotopy  type  (with  n  and  k  varying)  within  the  range 
of  finite  simplicial  complexes. 

We  may  think  of  as  measuring  controllability  in  the  graph  G,  whereas 

captures  the  adversary’s  ability  to  constrain  the  system.  The  full  meaning  of  the  factor 
is  still  a  research  question.  We  learned  a  lot  in  Section  10;  the  following  example  offers  further 
insight. 

11.4  An  Example  (Air  Travel  During  Thunderstorm  Season) 


Figure  23:  This  graph  contains  three  types  of  actions:  deterministic  (green),  stochastic  (red), 
and  nondeterministic  (blue).  The  graph  might  represent  some  poor  passengers’  potential  flight 
paths  during  a  day  of  thunderstorms. 

Consider  the  graph  of  Fig.  23,  which  might  represent  possible  air  travel  routes  between 
cities  in  the  United  States  during  some  thunderstorm-infested  July  afternoon.  The  graph 
includes  deterministic,  stochastic,  and  nondeterministic  actions.  Perhaps  deterministic 
actions  represent  flights  that  are  certainly  possible,  stochastic  actions  represent  flights  whose 
destinations  are  stochastically  determined  by  emerging  thunderstorms,  and  nondeterministic 
actions  represent  sets  of  possible  flights,  to  any  one  of  which  a  hopeful  passenger  will  be 
assigned  in  an  otherwise  unpredictable  fashion  (nondeterminism  is  Murphy’s  Law  in  action). 

Of  interest  is  the  set  of  cities  reachable  from  anywhere  in  the  system  and  whether  a 
passenger  might  become  trapped  in  an  endless  loop  trying  to  reach  some  particular  city. 

The  equivalence  relation  partitions  the  state  space  into  five  equivalence  classes.  Three 
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Figure  24:  The  two  nontrivial  equivalence  classes  of  the  relation  for  the  graph  of  Fig.  23, 
along  with  the  actions  establishing  full  controllability  within  each  equivalence  class. 

of  these  are  the  singleton  sets  {3},  {6},  and  {8}.  The  two  nontrivial  equivalence  classes  are 
{1,  2}  and  {4,  5,  7,  9, 10},  as  shown  in  Fig.  24. 

This  means  that  passengers  wishing  to  travel  between  cities  1  and  2  are  in  luck.  They  can  go 
directly  from  one  city  to  the  next.  Passengers  traveling  between  cities  in  the  set  {4,  5,  7,  9, 10} 
are  also  fairly  lucky.  They  will  certainly  get  to  their  destinations,  though  the  exact  routing 
may  not  always  be  certain  in  advance. 

Unfortunately,  there  is  a  convention  in  city  4,  towards  which  passengers  from  all  over  the 
country  are  traveling.  Many  will  not  make  it  today,  as  we  shall  see. 

The  left  panel  of  Fig.  25  shows  the  quotient  graph  for  the  graph  G  in  Fig.  23.  Each 

equivalence  class  is  represented  by  one  of  its  states.  The  source  complex  AG/^  of  this  graph 
appears  in  Fig.  26.  The  underlying  vertex  set  of  is  the  state  space  of  G/*->,  represented 

by  the  cities  {1,  3,  4,  6,  8}.  (Keep  in  mind  that,  for  instance,  state  4  really  represents  the  entire 
equivalence  class  of  cities  {4,  5,  7,  9, 10}.  Theorem  20  assures  us  that  the  capabilities  of  the 
graph  overall  are  captured  by  this  reduced  quotient  representation.) 

The  complements  of  the  maximal  simplices  of  AG  /<_>  describe  the  most  precisely  attainable 
goals,  as  implied  by  the  results  of  Section  10.  Since  (1,3, 4,  8}  is  a  simplex  in  AG/„,  this 
means  city  6  is  attainable  from  anywhere  within  the  graph  G/<-+  and  thus  from  anywhere  in 
the  graph  G.  The  right  panel  of  Fig.  25  exhibits  a  strategy  (in  the  quotient  graph)  for  attaining 
city  6.  Too  bad  the  convention  is  not  happening  in  that  city. 

Particularly  informative  are  the  minimal  nonfaces  of  A G/<_>.  The  complements  of  these 

_ * 

sets  are  maximal  simplices  in  AGy^,  meaning  they  are  the  maximal  goals  not  certain  to  be 
attainable  from  everywhere  in  the  graph  G/<->.  A q/^  contains  two  minimal  nonfaces,  namely 
{4,  6}  and  { 1 ,  3,  6,  8}.  The  complement  of  {1,  3,  6,  8}  with  respect  to  the  the  underlying  vertex 
set  of  A q/^  is  {4}.  Consequently,  city  4,  the  location  of  the  convention,  is  not  stochastically 
attainable  from  everywhere  in  the  system. 
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Figure  25:  The  left  panel  shows  the  quotient  graph  Cr/<->  obtained  by  collapsing  the  strongly 
controllable  subspaces  of  the  graph  G  in  Fig.  23  to  single  states  (not  shown  are  self-looping 
actions).  The  right  panel  shows  a  particular  strategy  for  attaining  state  6  in  this  quotient 
graph. 
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Figure  26:  Source  complex  Aqi<_>  of  the  quotient  graph  G/<-»  of  Fig.  25. 
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Figure  27:  The  graph  {G/*-*)\W  appears  in  the  left  panel,  with  W  =  {1,3,  6, 8}  being  a 
minimal  nonface  of  A q/<_>  (see  Fig.  26).  An  adversary  can  prevent  attainment  of  state  4  by 
selecting  transitions  as  in  the  right  panel.  The  system  then  has  full  controllability  within  W 
relative  to  this  adversarial  choice. 

The  left  panel  of  Fig.  27  shows  the  induced  subgraph  formed  by  flights  available  at  the 
equivalence  classes  of  cities  appearing  in  the  minimal  nonface  {1,3, 6, 8}.  Sure  enough, 
adversarially,  city  4  may  become  unreachable,  as  the  right  panel  of  Fig.  27  demonstrates. 

Remark:  The  reader  may  have  observed  that  in  the  right  graph  of  Fig.  27,  although  city  4 
becomes  unreachable,  the  system  can  move  to  any  city  within  the  minimal  nonface  {1,  3,  6,  8}. 
This  is  in  fact  generally  true:  if  some  set  W  is  a  nonempty  minimal  nonface  of  Ah  for  some 
graph  H,  then  the  system  has  full  controllability  within  W  relative  to  the  assumption  that  it 
does  not  exit  from  W.  In  other  words,  it  might  happen  that  the  system  leaves  W,  but  if  not, 
for  instance  if  an  adversary  makes  nondeterministic  choices  to  prevent  motions  out  of  W,  then 
the  system  can  move  to  any  state  within  W  from  any  other  state  in  W.  This  is  consistent  with 
the  observation  we  made  in  the  proof  of  Theorem  16  that  Aq\w  ~  for  any  such  W. 

12  Algorithms 

This  section  provides  algorithms  for  backchaining  from  a  goal  set  and  for  computing  a  graph’s 
strongly  controllable  subspaces.  With  these  tools  one  can  then  reduce  a  graph  to  its  potentially 
simpler  quotient  graph,  as  well  as  compute  the  source  complex  of  the  graph  or  of  its  quotient 
graph.  One  then  obtains  a  characterization  of  the  underlying  system’s  capabilities,  as  discussed 
in  the  previous  sections.  We  have  implemented  all  these  algorithms.  Sample  runs  produced 
the  figures  in  Section  11.4. 

We  do  not  provide  an  explicit  algorithm  for  computing  the  strategy  complex  of  a  graph. 
Our  homotopy  results  show  that  one  does  not  need  the  strategy  complex  per  se  in  order 
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to  understand  a  system’s  capabilities.  Moreover,  the  algorithm  Backchain  given  next  will 
produce  a  strategy  for  attaining  any  particular  goal  set  S,  or  determine  that  no  strategy  exists. 
However,  there  certainly  may  exist  strategies  in  a  strategy  complex  other  than  those  computed 
by  backchaining,  as  Fig.  5  suggests.  These  extra  strategies  may  be  useful  practically  as  backup 
strategies.  The  procedure  outlined  in  Section  11.2,  particularly  starting  with  “Now  suppose  ...” 
on  page  48,  allows  one  to  construct  such  strategies  individually  or  as  part  of  the  full  strategy 
complex. 


Algorithm  1  Backchain(G,S') 

Input:  A  stochastic  graph  G  =  (V,  A)  and  a  nonempty  subset  S  of  V. 

Output:  A  (possibly  empty)  set  {ui, . . .  ,Vk}  of  states  in  V  \  S  and  a  corresponding 

set  {<7i, . . . ,  c Jfc}  of  collections  of  actions  in  A ,  such  that  every  action  in  oi 
has  source  Vi  and  moves  off  the  set  {/ty, . . . ,  v^}.  for  i  =  1, ...  ,k. 

Moreover,  if  v  is  any  state  in  V\S  for  which  G  contains  a  stochastic  strategy 
for  attaining  S  from  v,  then  v  will  be  one  of  the  states  Vi  returned. 

Procedure: 

1.  Let  W0  :=  S. 

2.  For  z  =  1,2,...  until  done  do: 

(a)  Let  Vi  be  any  state  in  V  \  TFj-i  such  that  some  action  with  source  Vi 
moves  off  V  \  Wj_i.  If  no  such  vi  exists,  then  done. 

(b)  Otherwise,  let  cr*  be  all  actions  in  A  with  source  that  move  off  V\Wi-\. 

(c)  Let  Wi  :=  Wi_iUW- 

3.  Return  {ui, . . . ,  v^}  and  {a\, . . . ,  a^},  with  k  the  maximum  index  for  which  u* 
is  defined.  (If  there  is  no  such  k,  then  return  0  and  0.) 


Remarks:  When  Algorithm  1  is  done,  the  collection  of  actions  a  =  Uf=i  ai  is  stochastically 
acyclic,  constituting  a  strategy  for  attaining  V  \  {ui, . . .  ,Vk}.  In  particular,  if  Wk  =  V  when 
Step  2  is  done,  then  a  is  a  complete  stochastic  strategy  for  attaining  S  on  the  state  space  V. 
Conversely,  if  S  is  stochastically  attainable  in  G,  then  Wk  will  be  all  of  V .  as  the  proof  of 
Theorem  16  shows. 

Aside:  Applying  Algorithm  1  to  the  graph  of  Fig.  5  with  S  =  {3}  returns  one  or  the  other  of 
the  two  triangles  in  the  strategy  complex  shown  in  the  figure,  rather  than  merely  the  central 
edge.  This  is  because  Algorithm  1  grows  the  sets  Wj  one  state  at  a  time.  Traditionally,  one 
might  add  multiple  states  at  once.  Doing  so  would  then  produce  the  central  edge  as  the 
backchained  strategy  in  Fig.  5. 
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Given  a  goal  S,  the  next  algorithm  uses  backchaining  to  iteratively  winnow  a  graph’s  state 
space  V  down  to  the  maximal  subspace  IT  within  which  S  is  stochastically  attainable.  This 
procedure  will  be  a  useful  step  in  determining  a  graph’s  strongly  controllable  subspaces. 

(The  reader  may  wish  to  verify  that  the  is  correct,  that  there  really  is  a  single  maximal 
subspace  W  within  which  S  is  stochastically  attainable.) 

Algorithm  2  Strategy(G,5) 

Input:  A  stochastic  graph  G  =  (V,  A)  and  a  nonempty  subset  S  of  V. 

Output:  A  stochastically  acyclic  subgraph  H  =  (IT,  B)  of  G  such  that  S  C  IT  and 

Vj3  =  W  \  S,  with  IT  maximal  among  all  such  IT. 

Procedure: 

1.  Let  {t>i, . . . ,  Vk},  {<7i, . . . ,  <7fc}  be  the  results  returned  by  Backchain(G,5). 

2.  Let  IT  :  =  {ui, . . vk}  (J  S  and  <j  :=  Ui=i  ai- 

3.  If  IT  =  V,  return  the  graph  (IT,  a). 

4.  Otherwise,  recursively  call  Strategy((IT, C),  S),  where  C  consists  of  all 
actions  in  A  whose  sources  and  targets  lie  in  W. 

Given  a  stochastic  graph  G  =  (V,  A),  the  next  algorithm  computes  a  regular  directed 
graph  (T,  E)  such  that  ( v ,  u)  is  an  edge  in  E  if  and  only  if  G  contains  a  stochastic  strategy  for 
attaining  u  from  v,  with  v  distinct  from  u.  (See  again  the  definitions  on  page  31.) 

Algorithm  3  Reachable(G) 

Input:  A  stochastic  graph  G  =  (T,  A),  with  V  /  0. 

Output:  A  directed  graph  D  =  (V,E),  with  specifications  as  above. 

Procedure: 

1.  For  each  u  G  V,  let  Hu  :=  ( lT?i , Bu)  be  the  result  returned  by  Strategy^,  {it}). 

2.  Let  E  :=  j(u,  it)  G  V  x  V  v  G  Wu,  v  A  w}- 

3.  Return  (V,E). 
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The  next  algorithm  computes  the  strongly  controllable  subspaces  of  G. 

Algorithm  4  Subspaces  (G) 

Input:  A  stochastic  graph  G  =  (V,A),  with  V  A  0. 

Output:  The  set  {Wi, . . . ,  Wm}  of  equivalence  classes  of  for  G. 

Procedure: 

1.  Let  be  the  strongly  connected  components  of  the  directed  graph 

returned  by  Reachable(G).  See  [1]  for  an  algorithm  to  compute  these  components. 

2.  If  £  =  1,  return  {V\}. 

3.  Otherwise,  for  each  i  =  1, 

(a)  Let  Hi  :=  (Vt.Ci) ,  with  Ct  all  actions  of  A  whose  sources  and  targets  he  in  Vi. 

(b)  Let  Vi  be  the  result  of  calling  SUBSPACEs(iLj)  recursively. 

Return  (jf=1  Vi. 


Remarks: 

•  Correctness.  It  is  not  difficult  to  prove  that  the  previous  algorithms  correctly 
compute  output  as  specified. 

•  Runtime.  The  previous  algorithms  all  run  in  almost-reasonable  polynomial  time.  For 
instance,  the  slowest,  Subspaces(G),  has  time-complexity  0(|R|5|A|),  with  G  =  (V,A). 
No  doubt,  faster  implementations  exist. 

•  Quotient  Graphs.  Computing  the  quotient  graph  G/«->  from  a  graph  G  entails 
calling  Subspaces (G),  then  relabeling  sources  and  targets  of  actions.  Specifically,  if 
Subspaces  (G)  returns  {W\, . . .  ,Wm},  then  one  relabels  every  state  w  in  W%  by  some 
representative  state  for  IR,  with  i  =  1, . . . ,  m. 

•  Source  Complexes.  Computing  the  source  complex  Aq  of  a  graph  G  is  conceptually 
straightforward:  For  every  possible  goal  set  S,  one  calls  Backchain(G,5).  To  see  this: 

As  we  observed  earlier,  if  the  set  of  states  {vi, . . .  ,Vk}  returned  by  Backchain(G,5) 
is  equal  to  V  \  S.  then  S'  is  a  stochastically  attainable  goal.  Always,  {v\ . . . .  ,Vk}  is  a 
simplex  of  Ac-  Conversely,  any  simplex  of  Aq  must  show  up  as  the  start  region  of 
some  minimal  complete  stochastic  strategy  for  attaining  some  goal  S  in  G.  By  itself, 
without  other  actions  of  G,  that  strategy  necessarily  looks  like  a  backchained  strategy, 
as  can  be  seen  via  recursive  application  of  Lemma  15.  Consequently,  given  other  actions 
of  G,  backchaining  may  produce  a  different  strategy,  but  it  will  produce  some  complete 
stochastic  strategy  for  attaining  S  in  G. 

This  naive  algorithm  has  running  time  exponential  in  the  size  of  V .  One  can  tinker  with 
the  procedure  to  make  it  slightly  more  efficient,  but  the  exponential  nature  is  likely  to 
be  fundamental,  as  the  first  hardness  result  of  Section  14  indicates. 
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13  Realizability 


We  have  previously  indicated  that  finite  simplicial  complexes  and  stochastic  graphs  are 
essentially  equivalent  objects  from  a  topological  perspective.  In  fact,  this  equivalence  depends 
on  nondeterminism  not  stochasticity.  The  underlying  reason  is  that  finite  simplicial  complexes 
are  purely  combinatorial  and  thus  are  equivalent  to  finite  posets  [6,  75].  Nondeterministic 
graphs  alone,  without  stochastic  actions,  are  able  to  capture  all  finite  posets. 

This  section  makes  precise  these  comments  with  two  realizability  theorems.  The  first 
theorem  says  that  for  any  finite  simplicial  complex  E,  there  is  some  strategy  complex  isomorphic 
to  the  first  bary centric  subdivision  of  E.  The  second  theorem  says  that  E  may  actually  be 
realized  exactly  as  a  source  complex  of  some  graph.  In  both  cases,  the  generating  graphs  are 
nondeterministic. 

Reminder:  sd(E)  means  the  first  barycentric  subdivision  of  E  (Section  5.4). 

Theorem  21  (Realization  by  Strategy  Complexes)  Let  E  be  a  finite  simplicial  complex. 
There  exists  a  nondeterministic  graph  G  such  that  sd(E)  =  A^. 

Proof.  If  E  =  0,  then  sd(E)  =  0,  so  we  may  let  G  =  (0,  0). 

Otherwise,  define  G  =  (V,A)  as  follows:  V  consists  of  all  vertices  in  the  complex  sd(E) 
along  with  one  new  vertex,  _L.  A  contains  exactly  one  nondeterministic  action  Av  at  every 
v  £  V  other  than  at  _L.  Av  =  {(v,u)  \  u  £  V  and  {u, «}  is  not  a  simplex  of  sd(E)}.  In  other 
words,  Av  has  transitions  to  all  vertices  that  are  not  adjacent  or  equal  to  v  in  the  barycentric 
subdivision  of  E.  Observe  that  every  action  Av  contains  at  least  one  edge,  namely  a  transition 
from  v  to  _L,  so  this  definition  is  well-formed.  Moreover,  no  action  contains  a  self-loop. 

The  map  v  e- >  Av  is  a  bijective  correspondence  between  the  vertices  of  sd(E)  and  those 
of  A g-  In  order  to  establish  that  sd(E)  =  Aq,  we  need  to  show  that  this  correspondence 
preserves  simplices.  Definitionally,  both  sd(E)  and  Aq  contain  the  empty  simplex. 

(Aside:  If  E  =  {0},  then  G  =  ({ _L} ,  0)  and  A^  =  {0}  =  sd(E).) 

(a)  Suppose  0/d£  sd(E).  a  =  {iq, . . . ,  rq},  with  q  G  h  \  {T}.  Let  A*  be  the  action  of 
G  defined  at  Vi,  for  i  =  1, ...  ,k.  No  Vi  can  appear  as  the  target  of  any  Aj,  since  {uj, Vj}  is  a 
simplex  of  sd(E).  That  means  {Ai, . . . ,  A *.}  is  a  simplex  of  A^. 

(b)  Suppose  0/r£  A  q-  t  =  {Ai, . . . ,  A^},  with  each  Ai  £  A.  Let  vi  be  the  source  of 
A,;,  for  i  =  1, . . . ,  k.  Each  Uj  is  the  barycenter  of  some  nonempty  simplex  <jj  £  E. 

Consider  the  set  {vi,Vj}  formed  by  any  two  distinct  such  vertices.  This  set  must  be  a 
simplex  of  sd(E)  as  otherwise  {A*,  Aj}  could  generate  a  cycle  in  G.  Consequently,  either  dj  is 
a  proper  face  of  aj  or  vice-versa.  So,  without  loss  of  generality,  0  C  <ti  C  a 2  C  •  •  •  C  oq,  all 
inclusions  being  proper.  In  turn,  that  means  {iq, . . . ,  iq-}  is  a  simplex  of  sd(E).  □ 


Theorem  22  (Realization  as  Source  Complexes)  Let  T,  be  a  finite  simplicial  complex. 
There  exists  a  nondeterministic  graph  G  such  that  E  =  Aq-. 
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Proof.  If  E  =  0,  let  G  =  (0,  0).  Then  AG  =  0. 

Otherwise,  let  V  be  the  vertices  of  E.  If  E  is  the  complex  of  the  full  simplex  on  V ,  enlarge 
V  by  one  new  state,  _L.  Let  E  be  the  set  of  maximal  simplices  of  E.  Observe:  By  construction 
of  V,  V  0  E. 

Construct  G  =  (V.  A)  as  follows:  For  every  A  £  E  and  every  v  £  X,  define  the 
nondeterministic  action  Avx  =  {(u,u)  |  u  £  V\  X}.  Each  such  action  contains  at  least  one 
edge,  since  X  cannot  be  all  of  V.  Let  A  consist  of  all  such  actions. 

(Aside:  If  E  =  {0},  then  G  =  ({ _L} ,  0)  and  AG  =  {0}.) 

(a)  Suppose  0  /  X  £  E.  We  can  assume  without  loss  of  generality  that  X  £  E.  Consider 
the  set  of  actions  a  =  {Av_x  \  v  £  A}.  No  source  of  an  action  in  a  is  the  target  of  an  action 
in  a,  so  a  £  AG.  Thus  X  =  Va  £  AG. 

(b)  Suppose  0  /  X  £  Ag.  Let  a  £  AG  such  that  Va  =  X.  By  Lemma  15,  a  contains  an 
action  that  moves  off  X.  The  action  must  be  of  the  form  Avy,  for  some  Y  £  E,  with  v  £  Y. 
By  the  definition  of  “moves  off”,  v  £  X  and  all  targets  of  Avy  lie  outside  X.  By  construction, 
the  targets  of  Avy  are  all  the  states  of  V  \  Y.  That  means  AC  Y .  So  A  is  a  simplex  of  E, 
since  Y  is. 

Both  E  and  AG  contain  the  empty  simplex.  □ 

Remark:  The  underlying  vertex  sets  of  E  and  AG  may  differ  slightly,  but  the  simplices  in 
the  two  complexes  are  identical. 

14  Hardness 

This  section  presents  some  simple  hardness  results.  The  first  result  indicates  that  the 
exponential  computation  of  AG  described  in  Section  12  is  fundamental,  assuming  NP  and 
P  are  different  complexity  classes.  The  other  two  results  suggest  that  homotopy  equivalence  of 
strategy  complexes  is  either  a  stronger  topological  condition  than  one  needs  for  understanding 
system  capabilities  or  that  homotopy  equivalence  encodes  some  interesting  properties  about 
uncertain  systems  beyond  those  one  usually  asks  at  the  fixed  graph  level.  It  remains  a  research 
question  to  know  which  of  these  perspectives  is  correct. 

14.1  The  Difficulty  of  Determining  a  System’s  Precision 

The  first  result  has  practical  implications.  Given  an  uncertain  system,  it  may  be  very  difficult 
to  determine  just  how  precise  the  system  can  be. 

Lemma  23  (Precision  is  Hard)  Let  G  =  (V,  4)  be  a  stochastic  graph.  Determining  the  size 
of  the  smallest  stochastically  attainable  goal  is  NP- complete. 
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Proof.  The  proof  is  by  a  reduction  from  Independent  Set.  The  underlying  ideas  are 
related  to  the  realizability  ideas  from  Section  13. 

We  define  the  following  two  problems: 

IndepSet:  Given  an  undirected  graph  (V.  E )  and  an  integer  £,  is  there  a  set  /  Cf 
of  size  i  such  that  no  two  states  in  I  constitute  an  edge  in  El 

Precision:  Given  a  stochastic  graph  G  =  (V,A)  and  an  integer  k,  is  there  some  set 
S  C  V  of  size  k  such  that  G  contains  a  complete  stochastic  strategy  for 
attaining  SI 

IndepSet  is  a  known  ./VP-complete  problem  [49,  66]. 

I.  Precision  lies  in  NP: 

Given  S,  one  can  verify  in  polynomial  time  that  S  has  size  k  and  that  G  contains  a  complete 
stochastic  strategy  for  attaining  S,  using  Algorithm  1. 

II.  Precision  is  AP-hard: 

Suppose  (V,  E)  is  an  undirected  graph  and  l  an  integer.  Let  n  =  |  Vj. 

Define  a  nondeterministic  graph  G  =  (V(J  {_L},  A)  as  follows:  T  is  a  new  state.  For  every 
v  £  V,  A  contains  a  nondeterministic  action  Av  =  {(v,u)  \  u  =  1  or  (v,  u)  is  an  edge  in  E}. 
So  Av  transitions  to  every  state  adjacent  to  v  in  the  original  undirected  graph,  as  well  as  to  _L 
(ensuring  Av  is  well- formed). 

(a)  Suppose  /  is  an  independent  set  of  size  £  in  (V,  E). 

Consider  the  set  of  actions  a  =  {A*  |  i  e  /}.  No  source  of  an  action  in  a  is  the  target  of 
an  action  in  a,  by  definition  of  I.  So  a  £  A^.  That  means  G  contains  a  complete  strategy  for 
attaining  {_L}  (J  V  \  I,  which  is  a  set  of  size  n  +  1  —  t. 

(b)  Suppose  G  contains  a  complete  strategy  for  attaining  set  S  of  size  k. 

S  necessarily  contains  _L.  There  is  some  a  £  Ac  such  that  Va  =  V  \  S  and  |  |  =  n+1  —  k. 

Va  is  an  independent  set  in  (V.E).  To  see  this,  suppose  otherwise.  Then  a  must  contain 
actions  Av  and  Au  such  that  ( v ,  u)  is  an  edge  in  E.  That  would  mean  a  could  generate  a  cycle 
in  G,  a  contradiction. 

To  summarize:  (V,  E)  contains  an  independent  set  of  size  £  if  and  only  if  G  contains  a 
stochastically  attainable  goal  of  size  k  with  k  +  l  =  n  +  1.  One  may  construct  G  from  (V.  E) 
in  polynomial  time.  Thus  Precision  is  AP-hard.  □ 

14.2  Small  Realization  is  Uncomputable 

There  are  classic  deep  results  telling  us  it  is  algorithmically  impossible  to  recognize  the 
homeomorphism  or  homotopy  type  of  even  a  finite  general  simplicial  complex  [76].  These 
results  have  their  roots  in  work  on  the  algorithmic  impossibility  of  deciding  whether  a  word  in 
a  finitely  presented  group  is  trivial  [63] .  One  particular  implication  is  that  it  is  algorithmically 
uncomputable  to  decide  whether  a  finite  simplicial  complex  is  contractible  (this  follows  from 
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[64]).  We  present  two  consequences  for  nondeterministic  (and  thus  stochastic)  graphs. 

The  following  lemma  says:  Finding  the  smallest  graph  whose  strategy  complex  is  homotopic 
to  a  given  complex  is  an  uncomputable  problem.  Compare  this  with  the  realizability  results 
of  Section  13. 

Lemma  24  (Compact  Realization)  The  following  question  is  undecidable: 

Given  a  finite  simplicial  complex  E  and  a  nonnegative  integer  rn, 
is  there  a  stochastic  graph  G  with  m  actions  such  that  A  a  —  E  ? 

Proof.  Observe:  If  G  is  a  stochastic  graph  containing  a  single  action  A,  then  either 
Aq  =  {0}  or  Aq  =  {0,{A}},  depending  on  whether  A  contains  a  (non) deterministic  self¬ 
loop  or  not,  respectively. 

We  now  reduce  from  the  uncomputability  of  deciding  contractibility: 

Suppose  E  is  a  finite  simplicial  complex.  We  can  check  whether  it  is  trivial  (void  or  empty) 
easily.  Assuming  it  is  not  trivial,  we  ask  whether  there  is  a  stochastic  graph  G  containing  a 
single  action  such  that  A q  ~  S.  The  answer  is  “yes”  if  and  only  if  E  is  contractible.  □ 

14.3  Recognizing  Repercussions  is  Uncomputable 

Suppose  we  are  given  a  stochastic  graph  G  =  {V,  A)  that  somehow  does  not  satisfy  some 
criterion  we  desire.  We  wish  to  understand  whether  adding  a  particular  new  action  A,  with 
source  and  targets  in  V .  will  satisfy  our  criterion.  We  can  certainly  compute  the  strategy  or 
source  complexes  of  the  resulting  graph,  to  see  whether  any  larger  start  regions  or  more  precise 
goals  have  been  created.  What  we  cannot  do,  in  general,  is  decide  whether  the  homotopy  type 
of  the  strategy  (or  source)  complex  has  changed,  as  the  following  lemma  indicates. 

Lemma  25  (Detecting  Change)  The  following  question  is  undecidable: 

Given  a  stochastic  graph  and  a  new  action,  does  the  graph’s  strategy 
complex  change  homotopy  type  when  one  adds  the  action  to  the  graph? 

Proof.  Suppose  E  is  a  finite  simplicial  complex.  Again,  we  can  check  whether  it  is  trivial 
(void  or  empty)  easily.  Assuming  it  is  not  trivial,  we  construct  a  graph  G  much  as  in  the 
proof  of  Theorem  22,  except  that  we  always  add  two  new  states,  _L  and  T.  So  the  graph  looks 
like  G  =  (V,  A),  with  V  the  union  of  {_L,T}  and  the  vertices  of  E.  The  set  of  actions  A  is 
computed  as  in  the  proof  of  Theorem  22.  Observe  that  there  are  no  actions  at  either  _L  or  T. 
As  before,  one  obtains  exact  equality:  E  =  Aq- 

Now  consider  adding  to  G  a  new  action  A,  representing  a  deterministic  transition  from  T 
to  _L.  Call  the  resulting  graph  G.  Then  A~  is  a  cone  with  apex  A,  hence  contractible. 

•  E  =  Ag  —  Ag  (by  proof  of  Theorem  22  and  by  Theorem  14). 

In  summary: 

•  Ag  —  Ag  if  and  only  if  Ag  is  contractible  (by  construction). 

So  E  is  contractible  if  and  only  if  Ag  —  Ag. 

Recognizing  contractibility  is  undecidable.  □ 
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15  Topological  Thinking 

This  section  discusses  some  scenarios  in  which  topological  thinking  reveals  the  essence  of  a 
problem  and  its  solution. 

15.1  Topology  Precompiles  an  Existence  Argument 

Suppose  G  =  (V,  A)  is  a  stochastic  graph  with  associated  nonnegative  action  transition  times 
{5a}agA-  Let  S'  be  a  desired  stop  set  (goal  states)  within  V  and  let  T  be  some  nonnegative 
time. 

Suppose  for  every  u  in  V  \  S,  someone  has  produced  a  strategy  au  by  which  the  system  will 
converge  to  S  when  started  at  u,  with  worst-case  expected-convergence  time  for  u  no  greater 
than  T.  (In  other  words,  0  <  tu  <  T  in  the  solution  of  System  (1)  on  p.  22,  written  out  for 
au.  We  make  no  assertions  concerning  cru’s  times  tv  for  states  v  other  than  u.) 

We  ask  the  question:  Is  there  a  complete  stochastic  strategy  u  that  converges  to  S  from 
all  states  of  V  with  all  worst-case  expected-convergence  times  no  greater  than  T? 

Intuitively,  we  certainly  expect  the  answer  to  be  “yes” .  After  all,  one  can  imagine  a  meta¬ 
strategy  that  determines  the  initial  state  u,  then  executes  strategy  au.  However,  this  dispatch- 
based  strategy  is  not  a  strategy  in  our  sense.  Our  strategies  have  the  flavor  of  feedback-based 
control  laws,  that  is,  they  are  mappings  from  states  to  (sets  of)  actions.  The  question  therefore 
is  whether  we  can  flatten  the  dispatch-based  strategy  to  be  a  true  strategy.  Observe  the 
difficulties: 

Let  v  and  u  be  distinct  states  in  V  \  S. 

Issue  1:  It  could  very  well  be  that  au  contains  no  actions  at  state  v. 

Issue  2:  Even  if  cru  contains  one  or  more  actions  at  state  v,  it  could  be  that 
tv  >  T  when  we  solve  System  (1)  written  out  for  au. 

Issue  3:  The  two  strategies  au  and  av  may  contain  inconsistent  actions,  meaning 
that  <ju  U  ov  might  not  be  stochastically  acyclic.  For  instance,  one 
strategy  might  move  left  through  a  hallway  and  the  other  right. 

One  can  make  a  backchaining  argument  that  some  strategy  a  exists  for  attaining  S  from 
everywhere  in  V .  Arguing  that  among  all  such  strategies  there  is  at  least  one  for  which  all 
worst-case  expected  convergence  times  are  bounded  by  T  is  possible,  but  tedious. 
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There  is  a  short  topological  argument: 

•  Write  V  =  [n]  and  S  =  {n},  without  loss  of  generality. 

•  Consider  the  loopback  graph  GAn  =  ( V,  ) ,  as  in  the  proof  of  Theorem  1. 
Associate  transition  time  0  to  every  loopback  action. 

•  By  Lemma  10,  the  affine  cover  \JagA'  contains  all  sets 
{x  £  Rn  |  Xi  >  xn+  T}  and  all  sets  {x  £  R”  |  xn  >  Xj  }. 

Thus  it  contains  all  sets  {x  £  Rn  |  Xi  >  Xj  +  T},  with  i.  j  in  V. 

Using  Theorem  8  and  the  structure  of  dCr+e, 

=  u  k  n  bct+.  =  s"-2- 

AzA' 

•  By  the  remark  at  the  end  of  Section  8  (p.  35),  A q  contains  a  complete 
stochastic  strategy  for  attaining  S.  In  particular,  all  worst-case  expected 
convergence  times  of  this  strategy  are  bounded  by  T,  as  desired. 

The  intuitive  summary: 

•  Strategy  existence  depends  on  cover  connectivity  in  R™. 

•  Adding  strategies,  even  partial  or  pairwise  inconsistent 
strategies,  can  only  improve  cover  connectivity. 

Topology  has  precompiled  an  existence  argument:  Bellman’s  Principle  of  Optimality  shows 
up  as  the  union  of  open  sets. 

15.2  Topological  Analysis  of  Adversity 

In  looking  at  examples,  the  reader  may  have  begun  to  wonder  where  adversity  sits:  In  the 
stochastic  nature  of  actions  or  in  the  nondeterministic  nature  of  actions?  Topology  helps  us 
understand  an  answer  to  that  question. 

The  definition  of  moves  off  on  page  41  makes  explicit  a  difference  between  stochastic  and 
nondeterministic  actions:  A  stochastic  action  is  almost  the  same  as  a  set  of  deterministic 
actions.  The  following  definition  and  lemma  make  this  idea  precise: 

Definition:  If  G  =  (V.  A)  is  a  stochastic  graph,  let  det(G)  =  (V,  det(A))  designate  the 
nondeterministic  graph  in  which  every  stochastic  action  of  G  has  been  replaced  by  a  set  of 
deterministic  actions.  Specifically,  let  us  define  det(A)  as  follows: 

•  det(A)  contains  every  deterministic  or  nondeterministic  action  of  A. 

•  For  every  stochastic  action  A  £  A  of  the  form  A  =  {(v,piu\), . . . ,  (v,pkUk)},  det(A) 
contains  k  deterministic  actions,  each  consisting  of  a  single  edge  ( v ,  uf),  with  i  =  1, . . .  ,k- 
(As  usual,  we  permit  redundancies,  that  is,  distinct  actions  with  identical  edge  sets.) 
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Lemma  26  (Stochastic  Determinism)  Let  G  =  (V.  A)  be  a  stochastic  graph.  Then 

Ag  —  Adet(G)  and  Aq  =  A  det(G)- 

Proof.  The  first  assertion  follows  from  the  second,  by  Theorem  14. 

One  can  establish  the  second  assertion  by  using  Algorithm  1  and  Lemma  15.  □ 


Conclusion:  From  the  perspective  of  homotopy  type  or  from  the  perspective  of 

understanding  a  system’s  stochastically  attainable  goals,  one  may  study  det(G)  in  place  of 
G.  Fig.  28  provides  an  example,  demonstrating  homotopy  equivalence  of  Ag  and  Adet(G).  Of 
course,  convergence  times  in  the  two  graphs  may  very  well  be  different,  so  one  must  be  careful 
not  to  oversimplify. 


Figure  28:  The  top  row  shows  a  pure  stochastic  graph  and  its  strategy  complex.  The  bottom 
row  shows  the  deterministic  rendition  of  this  graph  plus  the  resulting  strategy  complex.  Both 
complexes  are  homotopic  to  S1. 


Pursuing  this  further,  we  can  ask  where  true  adversity  resides.  The  realizability  theorems 
of  Section  13  all  used  nondeterministic  graphs,  as  did  the  AP-completeness  result  of  Lemma 
23.  Suppose  G  is  a  pure  stochastic  graph,  in  the  sense  that  it  contains  only  deterministic  and 
stochastic  actions  but  no  multi-edged  nondeterministic  actions.  Then  Lemma  26  and  Theorem 
20  allow  us  to  conclude  that  Ag  is  homotopic  either  to  a  sphere  or  to  a  point.  Pure  stochastic 
graphs  simply  cannot  attain  the  richness  of  strategy  complexes  that  nondeterministic  graphs 
attain.  Said  differently,  true  adversity  comes  from  nondeterministic  adversaries.  Stochastic 
adversaries  can  slow  a  system  down,  but  do  not  have  the  ability  to  switch  between  limit  cycles 
in  the  way  nondeterministic  adversaries  sometimes  can  (limit  cycles  are  the  minimal  nonfaces 
of  Ag;  see  also  the  remark  on  page  55). 
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Some  further  remarks: 

Suppose  again  that  G  is  a  pure  stochastic  graph.  When  AG  and  Adet(Gj  are  homotopic  to 
a  sphere,  that  sphere  has  dimension  n  —  k  —  1,  where  n  is  the  number  of  states  in  G  and  k 
is  the  number  of  strongly  connected  components  of  det(G').  In  particular,  if  det(G)  can  be 
written  as  the  disjoint  union  of  its  strongly  connected  components,  then  AG  and  Adet(G)  will 
be  homotopic  to  a  sphere;  otherwise,  they  will  be  contractible,  as  follows  from  [44]  or  Theorem 
20.  Observe  that  k  is  no  greater  than  the  number  of  strongly  controllable  subspaces  of  G  and 
in  fact  can  be  less,  but  must  be  the  same  when  AG  and  Adet(G)  are  homotopic  to  a  sphere. 

When  passing  to  quotient  graphs,  let  us  discard  any  actions  that  become  cyclic 
(deterministically  self-looping).  Then  the  graph  det ((?)/<->  is  a  directed  acyclic  graph.  All 
the  actions  of  this  graph  together  are  acyclic  and  thus  the  complex  Adet(G)/^  consists  of  a 
single  simplex  (all  acyclic  actions  of  det(G) / <->)  and  its  faces.  It  is  empty  precisely  when  det(G) 
can  be  written  as  the  disjoint  union  of  its  strongly  connected  components. 

The  graph  G/+-+  is  similar,  yet  slightly  different.  As  we  observed  already,  the  state  space 
of  G/<r+  may  be  larger  than  that  of  det(G)/<->.  With  regard  to  strategies,  as  was  the  case 
for  Adet(G)/^,  AG/^  consists  of  a  single  simplex  (all  acyclic  actions,  now  of  G/<-+)  and  its 
faces.  A  difference  lies  in  certainty.  represents  a  collection  of  Markov  chains,  with  all 

states  either  transient  or  absorbing.  Whereas  in  det(G)/<-»  the  system  may  be  able  to  move 
purposefully  from  some  state  to  some  other  state,  in  £?/<->  it  may  only  be  able  to  do  so  with 
some  probability;  the  adversary  may  have  some  stochastic  choices  by  which  to  influence  the 
outcome  of  motions.  This  is  a  simple  Markov  Decision  Process  [67]. 

15.3  Topological  Thinking  in  Partially  Observable  Spaces 

This  section  shows  by  example  how  the  topological  characterization  of  task  solvability  may 
help  one  decide  whether  a  task  has  a  guaranteed  solution.  Throughout  this  section  all  graphs 
are  nondeterministic. 

Implicitly  in  the  analyses  thus  far,  the  system  has  always  known  its  current  state.  If  in  fact 
the  system’s  sensing  function  is  imperfect,  then  one  may  redefine  the  system’s  state  space  so 
as  to  obtain  perfect  sensing  in  the  redefined  state  space.  The  construction  is  straightforward 
[5,  54],  albeit  often  with  added  complexity. 

The  nondeterministic  version  of  this  construction  leads  to  a  derived  nondeterministic  graph, 
sometimes  called  a  knowledge  space  [23,  25].  A  sensorless  version  of  knowledge  space  appeared 
as  the  search  space  for  the  part  orienter  of  [29]. 

Intuitively,  the  knowledge  space  of  a  graph  is  a  new  graph  whose  states  are  sets  of 
potential  robot  locations  consistent  with  the  robot’s  sensing  and  action  history  at  runtime. 
For  completeness,  we  give  a  general  definition  here. 

Definitions  Suppose  G  =  (V,  A)  is  a  nondeterministic  graph. 

•  For  the  purposes  of  the  next  few  definitions,  we  assume  that  every  action  in  A  is  labeled 
with  a  name,  which  we  refer  to  notationally  as  A.  Names  of  actions  at  a  given  state  are 
unique.  However,  different  actions  at  different  states  might  have  the  same  name.  (Think 
of  a  global  control  with  possibly  different  effects  at  different  states.) 

We  write  Av  to  mean  the  action  in  A  whose  name  is  A  and  whose  source  is  v,  whenever 
such  an  action  exists. 


66 


•  A  sensor  on  V  is  a  covering  H  of  V  by  nonempty  subsets  of  V,  each  of  which  is  called  a 
sensory  interpretation  set.  Sensory  interpretation  sets  may  overlap. 

Intuition:  Whenever  the  system  is  at  some  state  v,  a  sensor  returns  some  value  whose 
interpretation  is  a  set  I  £  E  such  that  v  £  /.  (We  could  model  the  sensor  via  some 
intermediate  sensory  space,  but  we  will  simply  map  directly  to  interpretations  of  the 
sensor.  More  general  formulations  of  sensing  appear,  for  instance,  in  [5,  54].)  Sensing  in 
our  case  is  non  deterministic,  meaning  that  the  sensor  could  return  any  I  £  H  with  v  £  I. 

Observe  that  S  =  { W }  is  equivalent  to  no  sensing. 

•  Suppose  X  C  V  and  A  is  an  action  name.  The  forward  projection  of  X  under  A,  written 
Fa(X),  is  the  set  of  all  possible  locations  the  system  might  move  to  after  executing  some 
action  named  A,  given  that  the  system  starts  in  X.  In  order  for  this  definition  to  be 
sensible,  the  name  A  must  actually  mean  something  at  every  state  in  X.  Otherwise,  by 
convention,  Fa(X)  is  not  defined.  Formally,  if  Av  exists  for  every  v  £  X,  then 

Fa(X)  =  {u  £  V  |  (v,u)  £  Av  for  some  weX). 


•  Once  the  system  moves  from  X  under  A,  it  can  sense  again.  The  sensor  may  return  any 
sensory  interpretation  set  I  £  H  for  which  Fa(X)  f)  I  is  not  empty.  If  the  system  knows 
it  is  in  set  X  before  moving  under  A,  then  after  moving  and  sensing  a  particular  I,  the 
system  knows  it  is  somewhere  in  the  set  Fa{X)  f)  /. 


•  Given  G  and  H  as  above,  define  a  new  nondeterministic  graph  K  =  (U+,  A+)  as  follows: 

—  V+  consists  of  all  nonempty  subsets  of  V.  (Depending  on  H  and  A,  one  may  not 
really  need  to  construct  all  of  V+ ,  of  course.) 

—  For  every  X  £  V+,  and  for  every  action  name  A  such  that  Av  exists  for  all  v  £  A, 
A+  contains  a  nondeterministic  action  with  source  X  given  by: 


0  A  Y  =  Fa{X)  P|  I  for  some  /  £  H  j. 


K  is  the  knowledge  space  associated  with  G  and  E. 


Remark:  One  can  define  stochastic  analogues  of  K  for  pure  stochastic  graphs  G.  This  leads 
to  the  so-called  belief  states,  of  great  interest  in  Partially  Observable  Markov  Decision  Processes 
[54].  Some  caution  is  advised,  since  problems  quickly  become  intractable  [67,  58]. 

15.3.1  Inferring  Task  Unsolvability  From  Duality 

Recall  Theorem  16.  Failure  of  the  theorem’s  contractibility  condition  is  evidence  of  a  potentially 
inescapable  cycle,  appearing  as  a  minimal  nonface  of  A q,  indicating  that  the  graph  contains 
no  complete  guaranteed  strategy  for  attaining  set  S.  We  saw  an  example  in  Fig.  27. 

Now  consider  the  graph  of  Fig.  29,  which  might  model  a  robot  moving  in  a  building.  There 
are  two  corridors  (states  1  and  2).  In  any  one  corridor  the  robot  can  move  Right  or  Left. 
Atriums  connect  the  corridors  at  either  ends.  The  task  is  to  reach  one  particular  atrium  (state 
3).  Entry  into  the  corridors  from  the  other  atrium  (state  4)  is  imprecise.  The  gray  triangle 
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State  Graph  Imperfect  Sensor 


Inescapable  Cycle 


Figure  29:  A  graph,  an  imperfect  sensor,  and  a  potentially  inescapable  cycle  between  knowledge 
states  {4}  and  {1,2}  (indicated  by  the  solid  arrows  in  the  right  panel). 


Strategy  Complex  with  Perfect  Sensing 

4  — ►! ,  2  (Enter) 


Induced  Subcomplex 
(imperfect  sensing) 


Enter 


Right  Left 


Figure  30:  With  perfect  sensing,  the  strategy  complex  of  the  state  graph  of  Fig.  29  consists  of 
a  solid  triangle  joined  to  an  edge  of  a  solid  tetrahedron.  With  sensing  ambiguity  at  states  1 
and  2,  this  strategy  complex  collapses,  inducing  a  non-contractible  subcomplex  of  strategies 
in  knowledge  space  that  signals  a  potentially  inescapable  cycle. 
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in  the  strategy  complex  of  Fig.  30  constitutes  a  strategy  for  accomplishing  this  task,  assuming 
perfect  sensing.  The  strategy  is  to  Enter  from  state  4,  move  Right  from  1,  Left  from  2. 

Now  imagine  a  robot  controller  unable  to  distinguish  the  two  corridors  (states  1  and  2) 
based  on  sensing  alone.  The  task  no  longer  has  a  guaranteed  solution.  One  could  see  this  in 
a  variety  of  ways,  for  instance,  by  explicitly  constructing  the  robot’s  knowledge  space,  part 
of  which  is  shown  in  the  right  panel  of  Fig.  29.  Let  us  take  a  related  but  more  topological 
perspective. 

One  does  not  need  to  construct  the  knowledge  space  directly.  Instead,  one  can  reason  about 
strategies.  Consider  the  strategy  complex  of  the  original  graph  (left  panel,  Fig.  30).  As  we 
will  see,  this  complex  induces  a  non-contractible  subcomplex  of  strategies  in  knowledge  space 
(right  panel,  Fig.  30)  that  violates  the  contractibility  condition  of  Theorem  16. 

Let  us  focus  on  the  actions  at  two  key  knowledge  states,  namely  {4},  representing  certainty 
that  the  system  is  at  state  4,  and  {1,2},  representing  uncertainty  as  to  whether  the  system 
is  at  state  1  or  2.  The  tetrahedron  of  the  original  complex  describes  the  strategies  possible 
with  perfect  sensing  at  the  graph  states  1  and  2.  The  tetrahedron  collapses  to  an  edge  under 
sensing  ambiguity.  This  edge  represents  the  two  actions,  Right  and  Left,  possible  at  state 
{1,2}  in  knowledge  space.  In  the  original  complex,  only  a  portion  of  the  tetrahedron  joins 
with  the  action  Enter;  action  Right  at  state  2  and  action  Left  at  state  1  do  not  join  with 
Enter.  In  the  knowledge  space  complex,  the  edge  {Right,  Left}  consequently  cannot  join 
with  the  action  Enter. 

We  have  thus  exhibited  a  non-contractible  complex  describing  the  strategies  available  at 
a  subset  of  knowledge  space,  with  that  subset  lying  in  the  complement  of  the  goal  (state  3). 
This  means,  no  matter  what  the  surrounding  knowledge  space  might  look  like ,  there  can  exist 
no  complete  guaranteed  strategy  for  attaining  state  3  in  the  presence  of  control  uncertainty  at 
state  4  and  sensing  confusion  between  states  1  and  2. 

15.3.2  Hypothesis- Testing  and  Sphere  Suspension 

Definition:  A  suspension  of  a  complex  is  another  complex  formed  by  joining  each  simplex 
of  the  given  complex  with  each  of  two  new  vertices  [61,  39].  For  instance,  the  complex  in  Fig.  4 
is  a  suspension  of  the  complex  in  Fig.  2.  The  key  property  relevant  to  us  currently  is  that 
the  suspension  of  a  sphere  of  any  dimension  is  another  sphere,  of  one  higher  dimension.  For 
instance,  (a  globe  of)  the  Earth  is  a  suspension  of  the  Equator  by  the  North  and  South  Poles. 

Fig.  31  shows  a  variant  of  the  example  of  Fig.  29.  Once  again  there  is  some  control 
uncertainty,  once  again  the  sensor  cannot  distinguish  certain  corridors.  The  system  cannot 
move  reliably  to  state  3  using  pure  feedback  control  (that  is,  using  sensing  alone,  without 
history).  For  some  sensor  values,  neither  of  the  actions  Right  or  Left  will  make  progress 
toward  the  goal  3  at  all  graph  states  consistent  with  the  sensor  value. 

Fortunately,  this  time  the  confusable  corridors  lie  in  different  “wings”  of  the  building. 
The  sensor  stratifies  the  graph  into  two  subgraphs,  both  containing  the  goal  3.  Within  each 
subgraph  the  sensor  is  effectively  perfect  and  each  subgraph  contains  a  strategy  guaranteed  to 
attain  the  goal  from  anywhere  within  that  subgraph  using  sensor-indexable  controls  that  are 
well-defined  across  subgraphs. 

Whenever  a  sensor  stratifies  a  graph  into  subgraphs  in  this  manner,  there  exists  a 
hypothesis-testing  strategy  for  attaining  the  goal  from  anywhere  in  the  overall  graph. 
Hypothesis-testing  means:  The  system  assumes  it  is  in  one  of  the  subgraphs;  it  commands 
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State  Graph 


Imperfect  Sensor 


Figure  31:  An  imperfect  sensor  stratifies  a  graph  into  two  subgraphs  (the  “a  wing”  and  the  “b 
wing”),  over  each  of  which  sensing  is  effectively  perfect. 


actions  and  interprets  sensor  readings  as  if  it  really  were  in  that  subgraph,  but  also  verifies 
consistency  between  predicted  motions  and  observed  sensor  readings.  If  an  inconsistency 
occurs,  the  hypothesis  of  being  in  that  subgraph  has  been  falsified  and  the  system  moves 
on  to  another  hypothesis.  Intuitively,  this  strategy  eventually  converges  at  the  goal. 

Hypothesis-testing  is  a  strategy  in  knowledge  space,  but  one  does  not  need  to  construct 
knowledge  space.  In  general,  knowledge  space  may  contain  additional,  possibly  shorter, 
strategies.  Fortuitously,  for  the  example  of  Fig.  31,  hypothesis-testing  is  effectively  the  only 
strategy. 

There  is  a  short  topological  argument  that  hypothesis-testing  converges: 

Hypothesis-testing  amounts  to  repeated  sphere  suspension. 

Further  details:  The  graph  Ht  describing  motions  under  the  ith  hypothesis  is  equivalent  to 
the  subgraph  Gi  being  hypothesized,  except  that  some  actions  may  move  nondeterministically 
to  a  new  state  _U,  signaling  falsification  of  the  hypothesis.  We  also  add  an  “action”  from  _Lj 
to  the  goal,  as  explained  below.  Without  loss  of  generality,  every  action  of  Gi,  and  thus  Hi , 
contains  a  nondeterministic  transition  to  the  goal.  (Adding  these  transitions  does  not  affect 
the  existence  of  strategies  for  attaining  the  goal,  but  focuses  on  the  topologically  significant 
simplices  in  the  complexes.) 

The  relevant  loopback  complex  Tffi  of  Hi  is  then  a  suspension,  formed  by  joining  the 
relevant  loopback  complex  of  Gi  with  the  loopback  action  goal  — ►  _U  and  the  action 
_L i  — >  goal.  We  now  use  Theorem  1,  once  in  each  direction:  Since  Gi  contains  a  strategy 
guaranteed  to  attain  the  goal  from  any  state  in  Gi,  Tq,  is  homotopic  to  a  sphere,  of  the  correct 
dimension.  Since  F/^  is  a  suspension  of  it  too  is  a  sphere,  again  of  the  correct  dimension, 
proving  that  hypothesis-testing  converges. 

What  is  the  mysterious  action  _Lj  — >  goal  ?  It  is  the  inductive-hypothesis  that  there 
exists  a  strategy  for  attaining  the  goal  once  the  ith  graph-hypothesis  has  been  falsified!  (The 
base  case  is  similar,  except  that  there  is  no  need  for  a  state  _L The  final  graph- hypothesis  is 
certain.) 
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Remarks:  (1)  One  can  make  a  very  similar  argument  directly  at  the  graph  level,  further 
underscoring  the  parallel  between  spheres  and  task  solvability.  That  parallel  shows  as  well 
that  backchaining  in  nondeterministic  graphs  is  much  like  repeated  sphere  suspension.  (2) 
Hypothesis-testing  is  related  to  randomization  [23,  24,  25]. 

16  Conclusions 

16.1  Summary 

The  controllability  structure  Theorem  20  on  page  51  is  the  most  concise  description  of  system 
capabilities  we  have  currently.  Implicitly,  this  theorem  contains  as  special  cases  the  full 
controllability  Theorem  11  and  the  goal  attainability  Theorem  1.  These  three  theorems 
characterize  controllability  of  uncertain  systems  by  the  existence  of  spheres  of  a  specific 
dimension.  Theorem  16  provides  a  dual  statement  via  contractibility  of  subcomplexes.  That 
theorem  helps  one  understand  an  adversary’s  capabilities  in  terms  of  forced  limit  cycles. 

From  a  practical  perspective,  Theorem  14  on  page  42  is  the  most  useful.  It  suggests  that 
when  considering  homotopy  equivalences,  one  should  take  the  homotopy  simplification  of  a 
strategy  complex  down  to  the  state  space,  and  not  further.  That  rendition  provides  a  natural 
mechanism  by  which  to  compare  the  capabilities  of  different  systems  with  a  common  state 
space,  as  illustrated  by  the  design  example  of  Sections  9  and  10. 

16.2  Other  Results 

We  have  omitted  many  interesting  results  from  this  paper.  The  Contributions  section  (§1.6) 
mentioned  several.  For  instance,  the  number  of  nonempty  simplices  in  the  loopback  complex 
associated  with  a  particular  goal  is  even  if  the  goal  is  stochastically  attainable  and  odd  if  it 
is  not.  This  is  a  consequence  of  the  Euler  characteristic  being  even  for  spheres  and  odd  for 
contractible  spaces. 

There  also  exist  generalizations  of  the  results  presented,  by  reduction  to  them.  For  instance, 
perhaps  one  is  interested  in  whether  a  goal  is  attainable  from  some  initial  subset  of  the  state 
space  rather  than  from  the  entire  state  space.  One  can  reduce  this  problem  to  a  full  attainability 
question  by  modifying  the  graph  slightly.  In  the  nondeterministic  case  this  is  easy  to  see:  one 
merely  adds  a  nondeterministic  hyperjump  at  each  state,  that  jumps  to  every  state  in  the  initial 
region.  These  hyperjumps  will  never  be  needed  by  an  actual  strategy  in  the  original  system 
but  serve  as  a  convenient  reduction  tool. 

Similarly,  perhaps  one  is  merely  interested  in  reaching  a  goal  with  some  minimum 
probability,  but  not  necessarily  for  certain.  Again,  one  can  add  hyperjumps  at  some  states, 
assigning  strictly  positive  transition  times  to  these  hyperjumps  and  zero  transition  times  to  all 
other  actions.  One  thus  reduces  the  original  problem  to  the  problem  of  attaining  a  goal  with 
certainty,  but  now  with  a  bound  on  the  worst-case  expected  convergence  times. 

16.3  Open  Questions 

Our  original  goal  was  to  develop  a  topological  language  for  describing  system  capabilities.  We 
have  come  a  long  way.  Strategy  and  source  complexes  abstract  away  the  detailed  connectivity 
between  states  in  a  graph  while  retaining  information  about  overall  goal  attainability. 
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Consequently,  one  can  assign  topological  spaces  to  graphs,  much  like  labels,  that  succinctly 
summarize  the  graphs’  capabilities. 

One  wonders  whether  this  new  language  holds  more  expressive  power  than  we  have  used 
thus  far.  Our  key  theorems  are  not  yet  completely  “coordinate-free”:  The  number  of  states 
shows  up  in  the  dimensionality  of  the  spheres  that  assure  controllability.  Likewise,  we  find 
ourselves  working  with  source  complexes  on  specific  state  spaces,  rather  than  more  abstractly. 

Hidden  inside  this  observation  is  another:  Our  contractibility  results  tend  to  have  the  flavor 
of  collapsibility  (see  Section  5.2).  This  begs  the  question  whether  the  more  general  (and  more 
difficult)  form  of  contractibility  is  relevant,  and  if  so,  what  it  is  telling  us. 

Related  to  this  general  interpretation  of  contractibility  is  the  question  of  whether  our  results 
generalize  to  continuous  state  spaces.  As  we  observed  in  the  Introduction,  from  a  practical 
perspective  one  tends  to  work  in  discrete  spaces.  Nonetheless,  the  question  is  interesting 
intellectually.  For  instance,  imagine  a  sequence  of  finite  discrete  graphs  that  approximate  ever 
more  finely  a  continuous  space.  An  amusing  issue  here  is  that  finite-dimensional  spheres  are 
homotopically  different  from  points,  whereas  the  infinite-dimensional  sphere  is  homotopically 
equivalent  to  a  point. 
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21 
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23 
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41 

det(-) 

replacement  of  stochastic  actions  by  sets  of  deterministic  actions 

64 
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18 

5i;5n-2 

circle;  sphere  of  dimension  n  —  2 

15; 

16 

s,r 

simplicial  complex 

12, 

17 

(T,  T,  7 

simplex  in  a  simplicial  complex 

12, 
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nerve  of  a  collection  of  sets 

19 
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face  poset  of  a  simplicial  complex 

19 

sd(-) 

barycentric  subdivision  of  a  simplicial  complex 

19 

Ac 

strategy  complex  of  a  graph 

12, 

24 

Ag 

source  complex  of  a  graph 

42 

* 

Ag 

dual  complex  of  a  graph 

44 

^max(') 

maximal  worst-case  expected  convergence  time  of  a  strategy 

24 

ag 

subcomplex  of  AG  containing  time-bounded  strategies 

24 

VA,VB,Va,VT 

start  region  of  a  collection  of  actions 

22 

UA,UB,U+,U+ 

covering  sets:  open  subsets  of  Rn  associated  with  actions  A  and  B 

25 

N 

shorthand  for  { 1 , . . . ,  n} 

16 

s 

desired  stop  state  in  a  graph  (e.g.,  a  task  goal) 

16 

S 

set  of  desired  stop  states  in  a  graph  (e.g.,  alternative  task  goals) 

30 

Gk-S 

loopback  graph 

16 

ag_s 

loopback  complex 

16 

(M,  W) 

Markov  chain  M  with  support  W 

22 

* 

join  operator  (for  spaces  and  simplicial  complexes) 

46 

r^i 

isomorphic  (equivalence  relation  on  simplicial  complexes) 

17 

« 

homeomorphic  (equivalence  relation  on  topological  spaces) 

17 

homotopic  (equivalence  relation  on  functions  and  spaces) 

17 

<— > 

strongly  controllable  (equivalence  relation  on  states) 

49 

G /<-* 

quotient  graph  under  strong  controllability 

49, 

54 

G/W 

quotient  graph  formed  by  collapsing  W  to  a  point 

49 

Cri  dCr ,  dCx+e 

polyhedral  cylinders  measuring  convergence  times 

27 
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56 
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